Introduction

SMTP AUTH is a method for authenticating connections and negotiating security layers for the SMTP protocol. SMTP AUTH is defined in RFC 2554 and is based on SASL, the Simple Authentication and Security Layer. SMTP AUTH is supported in sendmail 8.10 and greater. The SASL layer is provided by the Cyrus-SASL libraries.

Goals and Background

Configuring sendmail to use SMTP AUTH is not a basic task. The goal of this document is to show how to successfully install and configure the necessary components to get SMTP AUTH working in sendmail. What we are trying to achieve by configuring SMTP AUTH is to provide a secure means of email relay.

In the early days of the Internet, all mail servers were open relays--that is, any client could relay mail through any server. This was not a problem as the number of hosts on the Internet were not great and the network was not used for commercial purposes. With the advent of easy Internet access for the general populace a problem arose. Spammers could send their unwanted junk mail through a third-party server, thereby shifting the costs of sending the spam to an innocent third-party.

Open relays had to be closed and a method was needed to allow legitimate users to relay mail through a server. One method is to use IP-based authentication whereby only certain IP addresses are allowed to relay. This works when the entire user base is static. Roaming users presented a problem, as their IP address would change based on where they were. Another method of authenticating roaming users is POP-before-SMTP. POP-before-SMTP requires a user to check their POP3 email before being allowed to relay mail. The server would record the incoming IP address of the POP3 request and then temporarily allow relaying from that IP.

POP-before-SMTP, while functional, has its drawbacks. As it is not an official Internet standard most MTAs (Mail Transfer Agents--the server software) and POP servers do not support POP-before-SMTP out of the box, requiring patches and additional configuration steps to enable. Also, there is the remote possibility that an unwanted party will be able to relay mail if the IP address that is authorized by the POP-before-SMTP system is shared among multiple users and computers.

SMTP AUTH provides a better solution--password protected authorization of the client who wishes to relay mail. Each SMTP session requires authorization and there is no possibility of accidentaly relaying unwanted mail. Unfortunately, not all methods of SMTP AUTH are supported by all MUAs (Mail User Agents--the client software) as will be seen.

Necessary Components

The examples that will be shown in this document were created on a system running Slackware 9.1. Slackware is a generic enough Linux distribution (and UNIX-like system in general) that these instructions should be easily transferrable to other Linux and UNIX flavors. Additional software is required to complete the installation, including sendmail, the GNU dbm (gdbm), and the cyrus-sasl implementation as shown in the table below. The configuration shown here will support the AUTH LOGIN mechanism. LOGIN is not necessarily secure, however, it is one of only two methods supported by Microsoft Outlook Express, by far one of the most popular mail clients in use today (this is merely an observation and should by no means be taken as an endorsement of MS Outlook Express by the author).

The AUTH LOGIN method utilizes BASE64 encoding of usernames and passwords. BASE64 is trivial to decode by anyone who might be sniffing network traffic, thereby revealing usernames and passwords. However, the goal here is to prevent unauthorized relaying through an SMTP server, not to provide high-levels of security to roaming users. Also keep in mind that most remote users are already sending their usernames and passwords in cleartext over the Internet via the POP3 protocol.

We will be configuring sasl to utilize the existing user accounts on the system by reading from the /etc/shadow file. There are other options available, including creating separate user accounts through the /etc/sasldb file.

Required Software

Package Name:	URL:
sendmail.8.12.10.tar.gz	http://www.sendmail.org
cyrus-sasl-1.5.28.tar.gz	ftp://ftp.andrew.cmu.edu/pub/cyrus-mail/
gdbm-1.8.3.tar.gz	ftp://ftp.gnu.org/gnu/gdbm/

Note that while there is a 2.x version of the cyrus-sasl implementation, it is not yet fully supported in sendmail 8.12.x, so be sure to download the most recent version of the 1.5.x tree.

Normally the Berkeley DB is used with cyrus-sasl, however, I found that while compiling against Berkeley DB 3.3.11, cyrus-sasl would not function properly and result in segmentation faults when running the saslpasswd command. A few vague references on mailing lists and newsgroups pointed towards inconsistencies in the Berkeley DB libraries causing these problems. Slackware 9.1 by default installs Berkeley DB 1.x, 2.x, and 3.x, so this may be part of the problem. In addition, there is a newer Berkeley DB 4.x which is not installed as part of the Slackware install.

Instead of using Berkeley DB, I chose to use the GNU DBM, or GDBM. While not normally as popular as the Berkeley DB, it is also part of the Slackware install and does work for our purposes, especially considering we will not be utilizing the sasldb functionality to create user accounts.

Important Assumptions

In order to ensure that this document is not overly long and tedious, I will be making a few important assumptions about the reader. First, we will assume that the reader has some basic knowledge of how to download, untar, and compile software on a UNIX system. Second, and most important, we will assume that the reader also has some knowledge of building sendmail.cf files from the M4 macros in a sendmail.mc file. The necessary macros that need to be added for SMTP AUTH to function will be presented, however, the subject of building a basic, functional sendmail.cf file is beyond the scope of this document.

It is assumed that the primary audience for these instructions is the administrator who already has a functional sendmail infrastructure and wishes to add SMTP AUTH capabilities to their site. In cases like this, most admins will already have a functional sendmail.cf file for their site (and hopefully the source M4 macros used to create it, in the form of a sendmail.mc file). Provided this assumption is correct, it is a simple matter to add the relevant M4 macros presented below to the existing M4 source to add SMTP AUTH capabilities.

Building the Cyrus-SASL Libraries

First, begin by making sure that gdbm is installed. If it was not installed as part of the Slackware OS install, either install the Slackware .tgz package from the CD, or compile the source code which can be downloaded from GNU.

The cyrus-sasl implementation must be compiled and installed. Download the tar file to a suitable location for building, such as /usr/src. It is assumed that the reader already knows how to expand gzipped tar archives and these obvious steps will be skipped. Take a moment to read over the INSTALL file, and also execute the configure script with the --help option to familiarize yourself with the options available.

Execute configure with the following options to configure the software in preparation for compiling:

# ./configure  --without-pam --enable-login --with-dblib=gdbm

Once configure is completed, review the output to make sure everything looks fine. Now, compile the software by simply running:

# make

# make install

Important Note: It is important that a symbolic link be created in /usr/lib to point to /usr/local/lib/sasl. The install will not do this for you, so it must be done by hand. It is also a good idea to run ldconfig at this point to update the cache used by the dynamic linker when compiling software.

# ln -s /usr/local/lib/sasl /usr/lib/sasl
# ldconfig

Configuring Cyrus-SASL

Each application that utilizes the SASL libraries must have a configuration file. These files are stored in the SASL library directory, in this example /usr/local/lib/sasl. For sendmail, the file should be called Sendmail.conf (Note the capital S in the name). This file will contain one line:

pwcheck_method: shadow

A blank /etc/sasldb file should be created at this time, even though it is not used in this specific configuration. The reason for this is that sendmail may complain on startup if the file does not exist. Make sure the file is only readable and writable by the root user and no group or other permissions are granted (chmod 600 /etc/sasldb).

Building sendmail

Download the sendmail tar file to /usr/src and untar it. In order to get sendmail to build against the SASL libraries, a site.config.m4 file must be created. This file resides in the devtools/Site directory, off of the root of the source directory. In our example, this file would be located at /usr/src/sendmail-8.12.10/devtools/Site/site.config.m4. The contents of this file must include at the least the following lines:

APPENDDEF(`conf_sendmail_ENVDEF', `-DSASL')
APPENDDEF(`conf_sendmail_LIBS', `-lsasl')
APPENDDEF(`confLIBDIRS', `-L/usr/local/lib/sasl')
APPENDDEF(`confINCDIRS', `-I/usr/local/include')

Next we will compile the sendmail binary. Sendmail is a bit different from other packages in that it does not utilize the now standard GNU autoconf method of building a Makefile. Change directories to the sendmail directory, off of the root of the source directory (/usr/src/sendmail-8.12.10/sendmail). Now, execute the following command to compile sendmail:

sh Build

sh Build install

Creating the sendmail.cf File

It will be necessary to add a few lines to the sendmail.mc file in order to generate a new sendmail.cf file to use SMTP AUTH. At the minimum, the following lines will need to be added to the .mc file:

define(`confAUTH_OPTIONS', `A')dnl
define(`confAUTH_MECHANISMS', `LOGIN PLAIN')dnl
TRUST_AUTH_MECH(`LOGIN PLAIN')dnl

sh Build sendmail.cf

sh Build install-cf

Testing

The sendmail binary itself can be tested using the following command:

sendmail -d0.1 -bv root | grep SASL

NAMED_BIND NETINET NETUNIX NEWDB PIPELINING SASL SCANF USERDB

The next test to complete is actually connecting to the daemon and verifying that your specified auth mechanisms appear correctly. Here is a successful output example with the desired string highlighted in red:

slacksite: {/home/slack/public_html} % telnet localhost 25
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
220 test.slacksite.com ESMTP Sendmail 8.12.11/8.12.10; Thu, 31 Mar 2005 23:59:14 -0500
ehlo localhost 
250-test.slacksite.com Hello IDENT:1007@localhost [127.0.0.1], pleased to meet you
250-ENHANCEDSTATUSCODES
250-PIPELINING
250-8BITMIME
250-SIZE 20485760
250-DSN
250-ETRN
250-AUTH LOGIN PLAIN
250-DELIVERBY
250 HELP
quit
221 2.0.0 test.slacksite.com closing connection
Connection closed by foreign host.

If both of these tests are successful then you should be able to fire up an appropriate mail client such as Microsoft Outlook Express, configure it to use SMTP authentication, and test it against the mail server.

If there is still a problem at this point, check the logs to see if there are any hints as to the problem. Messages for sendmail are usually logged to /var/adm/maillog or /var/log/maillog depending on your UNIX flavor. Common problems at this point would be incorrect file permissions on various SASL libraries or configuration files. The log entries will point out where these errors lie.

Conclusion

If all went well, you now have a fully functional SMTP server which supports the SMTP AUTH mechanisms LOGIN and PLAIN. Please keep in mind that PLAIN transmits in clear text and LOGIN is merely BASE64 encoded. Both of these methods will not increase your level of security other than to allow a server to no longer function as an open relay. Other methods should be considered to increase your level of password security, such as SSL Tunnels or STARTTLS, both of which are beyond the scope of this document.