Setting up sendmail on a firewall, Part 1

New features in sendmail 8.9.3 and how to build the binary

Summary
A tremendous amount of information is available about sendmail. But if you're in a hurry to just g et it working, you don't have the time to go through and digest it all. In this first installment in a three-part series on sendmail, Carole discusses its implementation on the firewall and focuse s on features new to sendmail 8.9.3. Next month she'll cover the particulars of the config file. < EM>(3,000 words)

One of the greatest features of sendmail is the extreme flexibility it provides the administrator. This is also one of its worst features, the reason being that it is so often misconfigured. Usually, a system administrator inherits a sendmail configuration file that he or she is afraid to touch, lest it should break. Over time, this file becomes hopelessly confusing, contradictory, and redundant.

I began writing this column with the intention of providing a simple, straightforward, "sendmail for dummies" approach. I ended up appreciating how Bryan Costales must have felt when he wrote his handbook, Sendmail for O'Reilly & Associates (see Resources): Nothing is simple in sendmail. I also learned to appreciate just how powerful and flexible sendmail really is. It's worth learning. It's also worth a few columns. I'll devote this month's column to discussing some of the background and new features in sendmail 8.9.3 as well as how to build the source code. Next month we'll look into the configuration file. If there is enough interest, I will add a third column for special situations and testing techniques.

Sendmail on the firewall?

Like most security admins, I've always been told it's a bad idea to run sendmail on the firewall. It's generally considered better to run something like smap instead. Indeed, I ran smap myself until I discovered a problem I couldn't fix with smap. The problem was that company email was forwarded from an internal mail gateway system to the firewall. The firewall could only rewrite the external header to say the mail came from "company.com." There was still an internal header that showed the mail was routed through "mailgate.company.com," complete with the mailgate internal IP address. I couldn't have the inside machine masquerade as company.com because there were other internal mail gateways it had to communicate with. (They couldn't all be company.com.) But because sendmail is extremely flexible, open source software, it's possible to add functionality to strip out the inside header. Also, many of the security risks with sendmail have to do with the fact that it's generally run setuid to root. It isn't necessary to run it this way on a firewall because there are no direct mail users on the firewall. For added security, I run sendmail in a chrooted cell with the program mailer using the sendmail restricted shell, smrsh.

Preliminaries

You'll also need the latest copy of the sendmail source code (sendmail 8.9.3 as of this writing), available from Sendmail.org. Verify the PGP signature! Some very useful links are included with the latest information.

Although it should go without saying, you need to be able to build the code using make and gcc or cc and have your environment variables set appropriately. It is strongly recommended that you use a system other than the firewall to compile and test the code. Remember that any system you use has to be running the same release of the operating system and have a compatible architecture. The firewall shouldn't have a development environment installed. If for some reason you have to use the firewall to compile, at least keep the development environment on a spare drive that you can unmount when you don't need it. Life with sendmail is much easier if you have the GNU utilities already installed because it is assumed in the makefile that groff is installed. You can get the GNU utilities (gcc, gunzip, groff, to name a few) from sunsite.unc.edu.

For the purposes of verifying the source (and not just for sendmail), you should have a copy of PGP (Pretty Good Privacy) installed (version 2.6.3 or higher). You can download a personal copy or buy one from Network Associates.

You can find links to these and other sendmail resources in the Resources section at the bottom of this article.

New features in sendmail 8.9.3

Relaying is denied by default in sendmail 8.9.x. While this might sound desirable, this feature will probably cause your mail to stop working because it will refuse to relay mail from your own internal systems. There are several ways to fix this. The easiest is to use the feature, relay_entire_domain. Refer to Sendmail.org for a full description of these new features.

Another major change to sendmail is that it is now picky about the permissions on the directories it uses. You can turn this off by using the DontBlameSendmail option. If you do, don't blame sendmail if you have security problems. I recommend fixing the directory permissions.

Default
Using the default options, you can simply create a file called /etc/mail/relay-domains that contains the names of the systems for which you want to accept relaying. For example, a domain known as "company.com" and also known as "othername.com" would have the following:

company.com
othername.com

• Advantage: If you only need to relay mail from a few systems, this is probably the simplest solution and the one I recommend for most installations. It prevents you from acting as a relayer, while allowing your mail to go through.

• Disadvantage: You have to keep the file updated, and you may have other requirements. If you're an ISP, you may have to allow relaying from quite a large number of domains. Also, every time you update this file you have to restart sendmail (kill -1 [sendmail_pid]).

promiscuous_relay
This feature allows relaying from anywhere (not recommended).

• Advantage: You don't have to worry about mail being denied.

• Disadvantage: Anyone can use your system as a mail relay. Use of this feature defeats the antirelay features of sendmail and is definitely not recommended.

relay_entire_domain
This feature allows any host within your domain as defined by the m class ($=m) to relay through your system. By default, this would be *.company.com.

• Advantage: You don't have to worry about mail within your domain being denied. Systems outside your domain can't use your system to relay mail unless they're defined in the relay-domains file or in the access database. If you only want to relay hosts in your own domain, you can use this instead of the file /etc/mail/relay-domains.

• Disadvantage: You may not want to allow other organizations in your company to route mail through your system. Also, if you're known by several domain names (popular with companies seeking product recognition), this won't buy you much.

relay_hosts_only
This allows relaying based on the individual host name, not the domain name.

• Advantage: Fine-tunes the permission for relaying by requiring a fully qualified host name rather than just the domain name.

• Disadvantage: Requires you to specify in either the file /etc/mail/relay-domains or the access database the host name of the system you're permitting to relay. For example, if I use this, my /etc/mail/relay-domains file would have:

  company.com
  
  mailgate.company.com
  
  othername.com
  
  mailgate.othername.com 
  

Mail blocking

accept_unresolvable_domains
By default, if the sender's domain cannot be resolved in DNS, the mail is rejected. For example:

MAIL FROM: <wkeys@nonexistent> 501 <wkeys@nonexistent>... Sender
domain must exist

Using accept_unresolvable_domains overrides this and accepts mail from any domain or IP address.

• Advantage: You may have to accept mail from IP addresses if the senders don't have their headers rewritten to come from a registered domain.

• Disadvantage: You can be spammed from anywhere. You may be better off listing known sites with this problem in the access database.

access_db
To use the access database feature, your system must support at least one map type such as NDBM (standard on most commercial systems such as Solaris) or the Berkeley database (Berkeley DB) 2.X. If you want to use Berkeley DB, you can obtain it from Sleepycat Software. If you install the Berkeley version, make sure you build sendmail with the NEWDB flag (and include the Berkeley DB libraries and include files).

The access_db feature causes sendmail to look in a database map file (by default /etc/mail/access.db) and decide whether to accept or reject mail from a particular user or site. You can even send a custom error message. This feature can also be used to control relaying permissions.

• Advantage: Really allows you to fine-tune who you will accept mail from. For example, I may not want to accept mail from domains that can't be resolved, but I want to make an exception for a particular domain or address. If yours is a large site, you may want the greater flexibility this can give you -- it's rather like a firewall rule base.

• Disadvantage: You have to build the database file and keep it updated. It can become rather complex -- it's rather like a firewall rule base. The more complex you make it, the harder it is to maintain.

accept_unqualified_senders
By default, if the sender's address isn't fully qualified, sendmail will refuse the connection. For example:

mail from: 
553 ... Domain name required

Use of this feature overrides the default so the connection will be accepted.

• Advantage: I recommend always using fully qualified addresses. However, on an internal mail gateway, you may not be able to control how the other local systems send you mail, and this will allow you to accept mail with unqualified sender addresses.

• Disadvantage: You lose some ability to track where mail is coming from. Don't use this on a firewall.

blacklist_recipients
This allows you to block incoming mail to accounts you don't want to receive mail by listing the account in the access database. The mail appears to be accepted but is actually dumped to /dev/null.

• Advantage: You may want to use this for "nobody" or "guest" accounts.

• Disadvantage: You have to set up the access database. This isn't much of a disadvantage, but it is another step.

rbl
If spam is a major problem for your site, you'll be interested in the Realtime Blackhole List. A list of known spam hosts is maintained at rbl.maps.vix.com. The rbl feature causes sendmail to check with rbl.maps.vix.com (or another RBL server if you specify) and blocks mail if the host is on that list.

• Advantage: You don't have to worry about maintaining your own list of spam hosts.

• Disadvantage: You're trusting that the RBL server is accurate. This may also cause a delay in accepting a connection while the RBL server is contacted.

relay_based_on_MX
If a host has an MX record that points to your site, this feature accepts and relays mail for them.

• Advantage: You don't have to add hosts to the access database if they have an MX record that points to you.

• Disadvantage: This seems like a lazy way out. It's probably quicker to look up systems in the access database than check for MX records. This will allow a third party to relay through you without your permission.

relay_local_from
This allows your users to relay mail though you if the domain portion of the sender address is a local host.

• Advantage: If you have users that belong in your domain but are routed through another domain, this will allow their mail to be relayed.

• Disadvantage: It's very easy to fake the sender address, and using this opens you up to spammers. I don't recommend it.

Building the sendmail binary

cp sendmail.8.9.3.tar.sig /usr/local/src [or wherever you keep your source]
cp  sendmail.8.9.3.tar.gz   /usr/local/src  	
cd   /usr/local/src
gunzip    sendmail.8.9.3.tar.gz    

You'll want to download the sig file so you can verify the validity of the tar file. Downlaod the PGPKEYS file from Sendmail.org in directory /pub/sendmail. This file contains the signature keys for Greg Shapiro and Eric Allman of Sendmail.org as well as the sendmail signing keys for 1997, 1998, and 1999. You must have the 1999 key for sendmail 8.9.3. You may want to include the older keys anyway if you're installing older versions of sendmail. PGP doesn't behave the same way in all versions. I used PGP Business 4.0.1 for Unix and discovered it doesn't like to have all the keys in the same file. It only wanted to install Greg Shapiro's key. I put each PGP key block into a separate file and then added them one at a time. You will want to certify the keys and check the fingerprints in the PGPKEYS file for each key against what the PGP software thinks the fingerprint is. If you're unfamiliar with PGP, you should refer to the documentation provided with your version of PGP. For Business 4.0.1, you would type the following command to add the key:

	pgp -ka /tmp/key1

/tmp/key1 is the name of the first key to add. Once you add the keys to your public key ring, you're ready to check sendmail's key. Type:

	pgp sendmail.8.9.3.tar.sig

You must have sendmail.8.9.3.tar in the same directory as the signature. PGP will verify the signature. If it's okay, it will tell you that the key used to sign the software is Sendmail Signing Key/1999. Once you have completed this step, you're ready to untar the software and begin installation.

	tar xvf sendmail.8.9.3.tar

This will unload the tar file and create a top-level directory, /usr/local/src/sendmail-8.9.3, which has everything you need to build and configure sendmail. You should review the README files for more information. In most cases, you can just run the build command without any configuration changes. Note that, by default, sendmail is compiled with DNS support. If you're not using DNS, sendmail will run very slowly until the DNS lookup times out. See the README file in the src directory for configuring compile-time options. To use the defaults, type:

cd  /usr/local/src/sendmail-8.9.3/src
./Build  -c

The -c option to build ensures a clean compile from scratch. Alternatively, you can run make from the top-level directory (/usr/local/src/sendmail-8.9.3) to build all the utilities provided with the release. It isn't necessary if you just want the sendmail binary, but you may want to use some of the included utilities such as makemap (for building an access database). Once the build has completed successfully, the binary will be in /usr/local/src/sendmail-8.9.3/src/obj.[ostype].[release].[arch], (obj.SunOS.5.6.sun4).

Next month

Acknowledgments

Disclaimer: The information and software in this article are provided as-is and should be used with caution. Each environment is unique and the reader is cautioned to investigate with his or her company as to the feasibility of using the information and software in the article. No warranties, implied or actual, are granted for any use of the information and software in this article and neither author nor publisher is responsible for any damages, either consequential or incidental, with respect to use of the information and software contained herein.