To enable end-user access to hosts in the Linux/UNIX environment, the old standbys have been telnet (for shell access) and FTP (to transfer files). These protocols have been around since the mid-1980s, and are defined by RFC 854 [rfc-archive.org] (telnet) and RFC 959 [rfc-archive.org] (FTP).

They're also very insecure. Both protocols conduct their communication “in the clear”, meaning that not only are the login credentials sent unencrypted, but all of the data sent back and forth are likewise open for anyone to read. Every keystroke, every password, every byte, can be silently intercepted and/or recorded.

Even within a “closed” or heavily-firewalled network, it is easy to overlook the dangers of unauthorized ARP- or ICMP-based re-direction, which would allow someone who had gained access to the network (legitmately or not) to conduct “man in the middle” (MitM) or similar attacks. Telnet and FTP offer no defenses against this, or any way to detect such subtle intrusions.

The original Internet environment was populated by a relative few, and relied as much on trust as anything else. More than 20 years have passed since then - the modern computing environment, for better or worse, is not inherently trustworthy, and insecure tools can be as dangerous as they are ubiquitous. Telnet and FTP are both. New, secure tools are needed.

...OpenSSH and rssh?

The Secure Shell protocol is actually a series of protocols, defined by several RFCs, including:

• RFC 4251 [rfc-archive.org] Secure Shell (SSH) Protocol Architecture
• RFC 4252 [rfc-archive.org] Secure Shell (SSH) Authentication Protocol
• RFC 4256 [rfc-archive.org] Generic Message Exchange Authentication for the Secure Shell Protocol (SSH)

OpenSSH [openssh.org] is probably the best-known implementation of the SSH suite, and arguably the one with the widest OS vendor adoption, being supported on platforms as diverse as IBM AIX, Novell NetWare, Sun Solaris and practically every Linux distribution.

Important!

The specifics of support for platforms vary by the platform. For example, the QNX implementation doesn't support post-authentication PrivSep (the PrivSep feature is described later in this paper). While NetWare has included OpenSSH for years, the NetWare console environment is not analogous to the *NIX terminal/shell environment; hence, rssh can't really function on the NetWare/OES-NetWare platform (thus, with regards to Novell's products, this paper is really only applicable to NetWare's successor, Open Enterprise Server, specifically OES-Linux; and SLES).

OpenSSH combines the shell/command-line functionality of telnet with the file transfer capability of FTP. Through its subsystem feature, its functionality can be extended even further. Using a single software stack for these services simplifies system management, providing a single point for configuration, authentication and logging.

Important!

This paper is based on the Portable version of OpenSSH, which varies in a number of ways, subtle and otherwise, from the OpenBSD version of the package. If you are using the OpenBSD version and encounter documentation conflicts between this paper and the man pages, consider the OpenBSD man pages as the more accurate source for that platform.

rssh [pizzashack.org] is an extension to the OpenSSH environment that allows the system administrator to more finely control access through OpenSSH. Designed to work specifically with OpenSSH, rssh is a restricted shell that can prevent or allow specific functions of OpenSSH, on a global or per-user basis.

...not “secure” telnet or FTP/S?

When considering implementation of telnet and FTP protocol variations that provide secure communications, the first thing one must ask is: “Which one?”

For example, just for FTP, there are at least two RFCs for providing “secure” FTP:

• RFC 2228 [rfc-archive.org] FTP Security Extensions
• RFC 4217 [rfc-archive.org] Securing FTP with TLS

Similarly, several RFCs cover options to “secure” telnet; among them:

• RFC 2946 [rfc-archive.org] Telnet Data Encryption Options
• RFC 3205 [rfc-archive.org] On the use of HTTP as a Substrate

None of these methods enjoys wide implementation or vendor acceptance. Further, this approach results in two software stacks for what are really very similar purposes, whereas OpenSSH is a single software stack for both activities. With the submission of RFCs for the SSH protocol, OpenSSH becomes just as “standard” as telnet and FTP have been.

...this paper?

The goal of this paper is to provide a basic roadmap for the installation and configuration of OpenSSH and rssh in a *NIX/Linux environment. Then, the paper will cover how this software combination can be leveraged to provide a secure communications framework. While functions, such as port forwarding and the ability to tunnel other protocols through SSH, are mentioned, those topics are outside the formal scope of this paper. Similarly, integrating OpenSSH with specific authentication back-ends (such as Kerberos) or additional security tools (such as TCP Wrappers) are also outside the scope.

It is important to keep in mind that these tools are not a magic security bullet. They will not protect against an account compromised by other means (e.g. someone learning a user's password, perhaps by social engineering), filesystem insecurity, attacks against other services (e.g. a webserver exploit) or insecure configuration of other tools or services.

OpenSSH [openssh.org] and rssh [pizzashack.org] are both Free/Open-Source Software (FOSS) packages. These softwares are among those that I prefer to build from source, so this paper covers those steps.

Common practice in the Linux/UNIX world is to install pre-built packages, usually provided by the platform vendor or distribution maintainer, rather than to build from source. Where a package is not part of the “official” offering, it is often available from a 3rd party (e.g. the package author, a site like SunFreeware [sunfreeware.com], or other independent repository). Many modern Linux/UNIX distributions include a pre-built OpenSSH. rssh is also available in package format, although it seems less common to find it already included in distributions (some versions of Gentoo and Debian have it).

Why, then, does this paper take the build from source approach?

When using a pre-built package, one must accept whatever compile-time configuration decisions were made by the package creator. These parameters may or may not be adjustable at run-time, and compile-time options selected by the package creator may or may not be appropriate to a specific environment. Building from source allows tailoring the software. As with any other system administration decision, weigh the factors in the environment and choose a course of action.

If you decide to use a pre-built package, skip past the Compilation and Installation sections - start reading at the Configuration section. You should follow the package installation documentation, and reference this paper for configuration tips. The current versions, as of this writing, are OpenSSH v4.7p1 and rssh v2.3.2.

This paper's assumptions, for those wishing to build from source, include the use of a modern Linux or UNIX (or UNIX-like) operating system, the ability to run shell scripts, compile C code (with GNU gcc or an equivalent), copy and/or link files, and set file modes and ownership. While some of these tasks require root privilege, you should only invoke that when you specifically need it (for example, most compilation steps can probably be accomplished as an unprivileged user). Additionally, this paper assumes that OpenSSL, and the zlib compression libraries, have already been installed and configured.

Every system admin has their own way of doing things, and their own sense of where to put files. This paper is written from my perspective on these issues, which may be different from the maintainers of the pre-built packages. Select whatever file location scheme for these tools that is appropriate to your environment, and if necessary translate my paths and locations into the scheme you select.

My personal environment is usually either SLES v9 (or later) with gcc v3.3.3 (or later), RHEL v4 (or later) with gcc v3.4.6, or Sun Solaris v8 (or later) with gcc v3.3.2 (or later). Other common tools include GNU make v3.8x and nano v2.0.x; however, I try to use a platform's stock development tools such as ld, ar, yacc and lex. I tend to create a sub-directory structure specifically for building add-on tools, generally /work. Make sure the partition where this is located has adequate room.

I usually install add-on tools, like OpenSSH and rssh, in a sub-directory off of /opt, and then use symbolic links in /usr/bin or wherever else might be needed. This allows me to control access better than if everything is dumped in /usr/local. I prefer symbolic links because I frequently make /opt its own partition, and I can “snap-in” a newer version of a tool with a few mv commands, since the link is merely a pointer to a path and file name (a hard link points to the inode entry and can't cross partition boundaries).

Administrators accustomed to working in a homogenous environment may wonder why this paper does not mention or advocate the use of specific tools for deployment beyond a single machine, or uses techniques that might seem problematic when viewed from a specific environment perspective. A reader might find themselves thinking “Why not just use <insert tool name here>?” or “That suggestion doesn't make sense in <insert specific environment name here>!!”

This paper is deliberately written for a generic audience, where a reader may be interested in applying the information presented in diverse environments, perhaps other than a typical Linux distribution or common UNIX variant. As a consequence, it offers ideas culled from a number of environments. The reader is encouraged to consider those ideas, techniques and tools that are applicable to their situation, and to ignore those that are not.

OpenSSH can be compiled with support for TCP Wrappers [porcupine.org], although doing so is not specifically covered in this paper. If TCP Wrappers is installed on the system but you decide to not compile OpenSSH with TCP Wrappers support, you can still leverage TCP Wrappers by invoking sshd via inetd (or xinetd). Consult the TCP Wrappers documentation for details.

Note to Solaris Admins: As of Solaris v9, Sun has been adding TCP Wrappers support to Solaris. The package name is SUNWtcpd.

Note to Linux Admins: Virtually every Linux distribution includes the OpenSSH package, and practically all of them link OpenSSH against TCP Wrappers. If you choose to compile your own OpenSSH (perhaps to get a newer version than the distribution maintainer provides), then linking against TCP Wrappers is recommended.

However might be appropriate for your environment, download the latest OpenSSH and rssh source packages. I download mine into /work/openssh and /work/rssh, respectively:

If your tar program includes the ability to decompress gzip-ed files, then you can use it directly; or you can call gzip as a separate step on the way to un-taring the files. Here, I'm showing the latter method, and writing three commands separated by semi-colons:

...and...

The commands will decompress the files, unpack the tarfile, then recompress the tarfile using the best compression offered by gzip. No sense wasting disk space leaving the uncompressed tarfile around. Or you can delete the tarfile. If you like and have it installed, substitute bzip2 [bzip.org] for gzip when re-compressing - the results tend to be better.

The source files now reside in a directory named very similar to the file from which everything was extracted. If you, like me, use the autocompletion capability of your shell, this can be annoying. So I usually change the directory name to something short:

Now autocompletion won't beep at me. I can get ready to build OpenSSH with:

Helpfully, OpenSSH uses the ubiquitous GNU autoconf configuration tool. The script configure will automatically examine your system, check dependencies, and prepare OpenSSH to be compiled with a built-in set of defaults.

However, since building from source offers flexibility, I recommend a few tweaks. For example, the default location of the OpenSSH binary is /usr/local, and as I noted above, I prefer /opt/openssh. You can see all the possible configuration options, and their defaults, with the “--help” parameter:

This will show you a lot of information, but won't actually configure or compile anything. What you might change from the defaults is largely dependent on your preferences and standards in your environment.

Within the context of this paper, I suggest the following defaults, which can be selected from the command-line when invoking the configure script:

Why these particular options and settings? Glad you asked that:

--prefix=/opt/openssh See my previous discussion about file locations.

--sysconfdir=/opt/openssh/conf Without this, the location of the OpenSSH configuration files and encryption keys would be in /opt/openssh/etc. As time-honored as etc is, I prefer the more-descriptive conf.

--with-zlib=/path/to/zlib This option simply tells the configure script where the zlib software is located. configure would almost certainly figure this out for itself, I'm just making sure.

--with-ssldir=/path/to/openssl Similar to the previous option, I'm simply saving configure to trouble of locating the OpenSSL files. Make the script earn its keep, if you prefer.

--with-pid-dir=/var/run When running, the sshd daemon will write its process ID in the file sshd.pid and place it in this directory. Actually, /var/run is the default if this option is omitted; it's being shown here for demonstrative purposes.

--with-privsep-path=/var/empty/or/alternate The concept of Privilege Separation is discussed in more detail below. This option merely defines the directory that will be used in the PrivSep process. The default is /var/empty, but it can be anywhere (within reason). This directory will be created by the make install command, if it doesn't already exist. If you create it by hand (or use a pre-existing location), the directory should be owned by root, mode 755, and not contain any other files or directories.

--with-privsep-user=user name Using this option, you can specify a non-privileged user name for Privilege Separation. The default value is sshd. This user name must exist or sshd will not run (the software can be installed without the user ID having been defined, but the install script will complain).

--with-mantype=man This option instructs configure to create documentation in man format. Other options include cat (for catman format) or doc. Set as you prefer.

At this point, the configure script should have been run, with any options you wanted. For the purposes of this paper, the assumption is made that it ran without errors, or that any issues have been fixed. Among other things, configure will generate Makefile in the same directory, and this is used for the rest of the process.

You're now ready to compile the OpenSSH suite:

This starts the compilation process. Troubleshooting compilation issues is outside the scope of this paper, so we'll proceed to the next stage.

If you've gotten to this point without any problems, the rest of the process should be equally trouble-free. To actually install the program files, you need to have root privilege because, among other things, you'll be setting SUID bits in file modes, and usually only root can do that. So, invoke su:

Now that you are privileged, install the program:

Again, troubleshooting installation error messages are outside the scope of this paper. When the script has run, it's a good idea to check the results, which should look like this (not all installation directories are shown):

So, what are all these files? And do you need them all? The answer to the second question is that you probably need most, but not all, of them.

First, let's look in /opt/openssh/bin. Here, you'll find secure implementations of many venerable *NIX programs client tools, as well as some new ones. scp is the Secure remote CoPy program, similar in function to rcp. As its name suggests, sftp is the Secure FTP client, which accepts many of the same commands as does a standard ftp client program. ssh (and its associated symlink slogin) is the Secure SHell client program that effectively replaces telnet, login, rsh, rexec and rlogin.

The programs ssh-add and ssh-agent are complementary, and form the foundation for a public-key-based single-sign-on framework, under which a user can automate authentication to other environments that support key-based authentication. And ssh-keygen and ssh-keyscan are tools for users to manage their own authentication keys. Each of these has its own man (or whatever documentation format you chose) page. Unless you wish to deny certain client tools to your user population, you need all the programs in this directory.

Next, the /opt/openssh/conf directory houses system-wide files. ssh_config is the configuration for the ssh program, but is ignored if the invocation of ssh includes the -F parameter. The file sshd_config is the configuration of the sshd server daemon. The server's private keys are in the files ssh_host_key, ssh_host_rsa_key and ssh_host_dsa_key. It is important that these files be protected (in the default installation they can only be read by root), as with them, anyone may impersonate your host. The server's public keys are in the files ssh_host_key.pub, ssh_host_rsa_key.pub and ssh_host_dsa_key.pub - these may be read by anyone. The file moduli contains large prime numbers for generation of Diffie-Hellman (DH) keys and was created as part of the installation (i.e. is unique to your system).

Security Note: If you subsequently perform an in-place upgrade of OpenSSH, the moduli file is not replaced if the install script detects an existing file. The extremely security-conscious (i.e. really paranoid) may consider periodic re-creation of the moduli file to thwart cryptographic analysis attacks that try to determine the primes used in key generation as a method to narrow keyspace. Of course, if the reader is actually worried about an attack on that level, then the reader has problems no technical paper can solve.

In /opt/openssh/share, we find Ssh.bin, which is an experimental Java applet for smartcard readers using OpenSC. If this is not needed, then you may safely remove this file and directory.

Finally, in the directory /opt/openssh/libexec, there are the programs ssh-keysign and sftp-server. The first is a helper program for host-based authentication, and is invoked by ssh when the EnableSSHKeysign directive is set to yes in ssh_config. The latter is the SFTP server subsystem invoked by sshd when a client requests that subsystem. SFTP is not provided by sshd without this subsystem, and subsystems in general are discussed later in this paper.

If everything is correct, and you've installed the OpenSSH client programs (e.g. ssh, sftp) into a location not normally in your $PATH (like /opt/openssh), then you'll probably want to link the files to /usr/bin or wherever is appropriate, like so (not all possible links shown):

Again, exactly what you install (or link) where is driven by how you administer your environment. This paper merely shows one possible way.

You can edit the sshd_config file that was installed, or write your own. The included file is moderately well documented. A more complete version is in the [Reference] section below.

It's not practical to review every possible keyword, but let's take a look at some of the more important ones, as they appear in the file provided in the Reference section:

Protocol 2 Prior to OpenSSH v4.7p1, the default configuration file for sshd supported both SSH v1 and SSH v2 protocols. Starting with OpenSSH v4.7p1, SSH v2 is the only configured default. If you install the newer version atop an older installation, the configuration is not changed. SSH v1 has a number of security weaknesses and is considered deprecated (although a number of devices - like routers - that support SSH access only support SSH v1). Unless you have some insurmountable technical challenge that prevents you from requiring clients accessing your SSH server to always use SSH v2, do not enable SSH v1 support.

AllowTCPForwarding yes Known as “the poor man's VPN”, TCP Forwarding allows TCP packets from an SSH client machine to be redirected, or forwarded, to other ports on the host running the SSH server. They can also be redirected to remote hosts. Disabling TCP Forwarding does not prevent users with shell access from setting up their own forwarders. To a certain extent, you must be able to trust users with shell access. As a rule, never allow TCP Forwarding on SSH servers that allow anonymous access, such as an Anonymous CVS host. This is because your network can then be probed, with an attacker using the anonymous access to redirect packets to various hosts within your network. Shell access is not required to abuse TCP Forwarding - any sshd-delivered service can be leveraged. In a situation where you must permit TCP Forwarding on a host that also has anonymous access, then use tools such as TCP Wrappers to strictly control the behavior of sshd. As noted previously, TCP Wrappers is out-of-scope for this basic paper; you should consult the OpenSSH and TCP Wrappers documentation concerning using TCP Wrappers to exercise control over SSH. If you are able to prevent users from editing their authorized_keys file (which would generally mean that you prohibit them from writing at all to their home directory; or you use a global authorized_keys file that you control), then it's possible to use options on the keys in the file to limit what the users can do with respect to forwarding TCP packets. Finally, introduced as an sshd option in OpenSSH v4.4, the PermitOpen directive affords a measure of granularity over TCP Forwarding. Consider limiting any TCP Forwarding you enable using this option.

GatewayPorts no If enabled, this option allows the SSH server to listen for forwarded ports on any interface. Normally, sshd will only listen for forwarded ports on 127.0.0.1, the loopback interface. This means that only programs running on the SSH server can have their TCP packets forwarded to an SSH client. Other hosts cannot have their TCP packets forwarded through the SSH tunnel from the server to the client. If you set this option to yes, then sshd can be told to listen on other interfaces for TCP packets hitting forwarded ports, and forward them to the SSH client that requested the packet forwarding (this arrangement, where the server forwards to the client, is sometimes called reverse forwarding). SSH clients request this functionality with the -R parameter when setting up forwarding. TCP Forwarding presents significant security issues, and Gateway Ports magnifies those issues by allowing any host to forward packets through the SSH tunnel. While access to this functionality can be controlled with TCP Wrappers, you should be sure you have thoroughly researched the effects of this option, and understand the security ramifications of allowing TCP Forwarding in general.

MaxStartups 5 Using this option, you limit the number of unauthenticated connections that the SSH server will accept at any one moment in time. The default is 10, and you should adjust this higher or lower as your environment requires. Authenticated connections do not count against this limit. When the limit is reached, further connection requests are summarily refused, until a pending connection authenticates, times out, or is dropped (perhaps due to too many authentication failures). If the limit is set to 0, then there is no limit on simultaneous unauthenticated connections, and connections are always accepted (until the server runs out of resources). The limit can also be expressed as A:B:C, where A is a lower bound, C is an upper bound, and B is a percentage. In this situation, when the number of unauthenticated connections reaches A, the next connection has a B% chance of being rejected. As the number of unauthenticated connections grows above A, the chance of rejection of a given new connection grows in a linear fashion, roughly expressed by the equation ((100 - B)/(C - A))% per additional connection. When the number of unauthenticated connections reaches C, the chance of rejection of a new connection is 100%. This remains true until an existing pending connection authenticates, times out, or is dropped.

PermitRootLogin no When set to no, the root account is not allowed to login, even if it is listed in AllowUsers. Any account with a UID of 0 is affected by this option, not just the account named root. There are also several caveats that are important to understand. First, listing root in DenyUsers - or a group of root in DenyGroups - will block root at connection time. In contrast, if the only impediment to root login is this option, and proper credentials are provided, the authentication process actually succeeds; the session is simply immediately terminated. This difference is important in that if you want to absolutely block remote root access, then doing so with DenyUsers/DenyGroups is the more sure method. Next, there exists a poorly documented loophole with this option. RSA (key-based) authentication is allowed to succeed for root, regardless of this option, if a command has been specified that matches a command in the account's authorized_keys file. Again, if you really want to block root, use a specific listing in DenyUsers. Finally, using without-password instead of no restricts root login to authentication methods other than the account password, such as key-based authentication. This prevents someone who possesses the root password, but not the account's private key, from logging in. If you simply have to allow remote root login, and your hosts must also be available from untrusted/untrustable networks (such as the Internet), then consider architecting a solution with multiple instances of the sshd server, augmented with host-based firewalls or tools like TCP Wrappers, where the instance of sshd that allows remote root access is only available to hosts on a trusted network.

DenyUsers AllowUsers DenyGroups AllowGroups These options allow you to specify either user accounts, or groups in which user accounts are members, that are either specifically denied or permitted to login via SSH. These options are evaluated in the order listed above, regardless of the order in which they appear in sshd_config. Evaluation of these options is by string comparison to user/group names (not UID or GID), and wildcards (* and ?) may be used to form patterns for comparison. For example, jo* would match john and joanne, but not jack. Similarly, t?m would match tim and tom, but not tommy. When a user attempts to login, their user name is compared against the list (or patterns) in DenyUsers. If DenyUsers is not in sshd_config, or if no match is found, then the user name is compared to AllowUsers. If that option hasn't been specified, then login is allowed to proceed to the next step. If it has been specified, then if the user name cannot be matched, the user will be denied login; otherwise, login is allowed to proceed. If the user name is not denied, then sshd retrieves the groups (Primary and any Secondary) to which the user ID belongs. These are all compared against the list of group names and/or patterns in DenyGroups, if it is defined in the configuration. If a match is found, then the user is denied login; if DenyGroups is not defined, or no match was found, then the AllowGroups option is checked. If AllowGroups is not defined, or is defined and any group of which the user is a member can be matched, then login proceeds. If AllowGroups is defined but no match is found, then login is denied. The general approach of these options is to deny login if any reason can be found to do so. It's important to note that if AllowUsers or AllowGroups is specified, then any user or group name not explicitly listed (or which cannot be matched to a pattern) will be denied login. When deciding how to approach access control with these options, consider carefully the ways in which they interact. Generally, it's easier to implement and understand a configuration that uses either DenyUsers/Groups or AllowUsers/Groups, but not both.

PermitUserEnvironment no This option controls whether or not sshd will honor environment= parameters on keys in the user's .ssh/authorized_keys file, or read the user's .ssh/environment file. If set to no (the default), then the parameters and environment file, if they exist, are ignored. If this option is yes, then the file is read, if it exists, and any environment= parameters on authentication keys are honored. Within the context of the configurations and system design presented by this paper, this option should never be enabled, as it can allow users to bypass access restrictions. Also, enabling user control of environment variables will break rssh security.

Compression delayed Using this option, the administrator can control when, or if, sshd will honor client requests for data compression across the link. The default value, delayed, instructs the SSH server to ignore client requests for compression until after authentication is successful. This prevents unauthenticated connections from attempting to exploit data compression libraries (for example, as shown in CVE-2005-2096 [nist.gov]), while still allowing authenticated connections to request compression. Using no means that all client requests for compression are ignored; while yes means that any client request for compression is honored, even for unauthenticated connections. While data compression can be useful and significantly enhance performance, especially across slower links, it's not always appropriate. SSH connections from the local LAN rarely benefit from compression, and in some cases, such links will actually get worse performance if compression is used. Important: Understand that using Compression delayed may cause connection negotiation p