 Milter
Frank Wiles
Most people realize that email is one of the most revolutionary
inventions the world has seen. The ability to send a quick text
message, document, image, or program to someone across the globe,
in seconds and with essentially no cost, has transformed communication.
Email server software is built and designed to send and receive
email and do not much else. Other than providing some simple sanity
checks and anti-spam features that spammers easily circumvent, email
server software is pretty boring stuff.
Many sys admins struggle to get their email servers to handle
more challenging tasks such as spam filtering, virus scanning, and
corporate email policy compliance. Sendmail is the most popular
email server software in the world and, as you would expect, has
a very powerful feature to help sys admins and programmers tackle
these more difficult chores. Milter (a.k.a. libmilter) is Sendmail's
answer to features that Sendmail does not itself currently support.
Need spam or virus filtering? Need to append a corporate disclaimer
to all outbound emails? Need to count the number of times a message
that contains the word "rhubarb" is sent? Milter is your answer.
Email travels via the Simple Mail Transport Protocol (SMTP). This
protocol includes several steps, but a typical message uses the
following four commands: HELO, MAIL FROM, RCPT TO, and DATA. HELO
tells the receiving server the hostname of the sending server and
starts the transaction. The sending server then issues the MAIL
FROM command, which states who the message is from. This is followed
by RCPT TO commands for each recipient of the message. Finally,
the DATA command is issued, which is the where the header and body
information of the message is transferred.
Milter provides the ability to have one or more programs inspect
a message and react at any point in the SMTP transaction. These
programs can log information, accept/reject the message based on
complex criteria, or anything else you could possibly want to do
with a message.
Configuring Milter Support
Some distributions of Sendmail, such as Red Hat and Fedora Linux,
have Milter support compiled in by default. You can test your existing
setup for Milter support by issuing the following command as root:
sendmail -d -bv root | grep MILTER
If that command returns a line of output, you already have Milter
support in your installation of Sendmail. If not, you'll need to compile
in support by adding the following lines to site.config.m4 file:
APPENDDEF('conf_sendmail_ENVDEF', '-DMILTER')
You should then build libmilter by running the ./Build command
in src/libmilter and recompile Sendmail. It's wise to test this new
Sendmail with the above command to make sure that Milter support was
properly added.
To start using Milter, you need to configure Sendmail to start
talking to one or more Milter processes, which is accomplished with
the INPUT_MAIL_FILTER macro in your sendmail.mc file. If you have
a Milter process running on the same server as your Sendmail processes,
then you would use a configuration line similar to this:
INPUT_MAIL_FILTER('myfilter', 'S=local:/var/run/myfilter.sock, F=R')
This tells your Sendmail server that it needs to talk with the locally
running Milter process "myfilter", which has a Unix domain socket
at /var/run/myfilter.sock. The "F=R" option tells Sendmail that it
should reject the SMTP connection if it cannot communicate with the
myfilter. This option should be used for filtering viruses, for example,
but you might want to consider leaving out that option for something
like spam filtering. By doing this, if there were a problem with your
filter, messages would pass right through Sendmail as if nothing were
wrong (granted, without any filtering).
To use a filter on a remote server, configure Sendmail like this:
INPUT_MAIL_FILTER('myremotefilter', 'S=inet:3333@remote.hostname.com')
This tells Sendmail to use the Milter process running on port 3333
on the server remote.hostname.com. This option is useful if you have
several mail servers but want to centralize your mail filtering.
Using Existing Milter Programs
Once you have Milter support in Sendmail, you'll need to set up
some Milter processes. There are several open source filters available
for different tasks. Some of the more popular include milter-spamc,
which gives a Milter interface to the popular SpamAssassin software;
MIMEDefang, which is a powerful mail filtering framework; and milter-greylist,
which implements grey listing on your email server. Links to all
of these projects can be found at http://www.milter.org.
Now I'll describe how to set up milter-spamc so that you can start
filtering your email for spam via Milter. Assuming you already have
SpamAssassin (http://www.spamassassin.org) set up with its
daemon (spamd) running on the local server, this is very easy to
do. You'll need to download the source for milter-spamc and libSnert
from http://www.snert.com/Software/milter-spamc/. It's then
just a normal ./configure, make, make install
for both packages.
Configuring your Sendmail software to use milter-spamc is simply
a matter of adding the following lines to your sendmail.mc and rebuilding
your sendmail.cf from it:
INPUT_MAIL_FILTER( 'milter-spamc',
'S=unix:/var/run/milter-spamc.socket, T=S:30s;R:2m')
define('confMILTER_MACROS_CONNECT', confMILTER_MACROS_CONNECT',
{client_addr}, {client_name}, {client_port}, \
{client_resolve}')
The "T=S:30s;R:2m" tells Sendmail to temp fail messages if it cannot
connect to milter-spamc, send data in 30 seconds, or read data back
in two minutes. Then, simply start the milter-spamc process with the
appropriate command-line options for your setup. You'll need to reference
the milter-spamc documentation to find out what options you want.
Be sure the socket location you tell milter-spamc to use matches the
location you've defined in your sendmail.mc; otherwise it will not
work.
Restart Sendmail and let the filtering begin. If you get a "Socket
write error: connection refused" error message, make sure your spamd
is running and that you've given milter-spamc the proper arguments
to know where to find it.
Writing Your Own Custom Filters in Perl
Because Milter processes run as a daemon and are not executed
for each mail transaction, Perl makes a great platform for writing
filters. Note, however, that all of your filters must be thread
safe and only use thread-safe CPAN modules if you use any at all.
To write Perl programs that use the Milter interface, you must
install the Sendmail::Milter module from CPAN.org. Because the programs
are multi-threaded, your Perl interpreter must have thread support
compiled in. Make sure you are using at least version 5.6.1 of Perl
and that it has been compiled with -Dusethreads. You can
check for this by running Perl -V.
To build Sendmail::Milter, execute the following commands:
perl Makefile.PL /path/to/sendmail-sources /path/to/sendmail-objects
make
make install
The arguments to Makefile.PL need to be the paths to your Sendmail
source tree and Sendmail build locations, respectively, specifically
the libmilter information. On my Fedora Linux system, I used:
perl Makefile.PL /usr/include/libmilter /usr/lib
Now that Sendmail::Milter is installed, let's write a simple filter
that can be used to append a company disclaimer to all email messages.
See Listing 1.
While this may look very complicated, it's actually quite simple.
The code inside the BEGIN block attempts to configure the filter
to use the connection information provided in your Sendmail configuration.
It also tries to clean up any socket files that were left from a
previous run of the software and registers the callbacks that you've
written with Milter.
In a filter, you can define a callback for any of the following
parts of a message transaction:
| Callback Name |
Description |
| connectn |
The start of the SMTP connection |
| helo |
HELO |
| envfrom |
MAIL FROM |
| envrcptO |
RCPT T |
| header |
Email headers |
| eoh |
Called at the end of all the headers |
| body |
Message body |
| eom |
Called at the end of the message body |
| abort |
Called when the SMTP transaction is aborted |
| close |
Called at the close of the session |
We store references to our callback functions in a hash called
%callbacks, which then passes to Sendmail::Milter::register()
to let Sendmail know what functions to call at each part of the
transaction.
The "return (SMFIS_CONTINUE)" you see at the end of each function
tells Sendmail to continue processing the message while still calling
the subsequent callback functions. Other options for this are SMFIS_REJECT,
which rejects the message, or SMIFS_ACCEPT, which accepts the message
and does not call any further callback functions.
We also pull $ctx into each function; this is always the
first argument passed by Milter into your code. It contains context
information about the current SMTP transaction and can be used to
pass information between callback functions. Using $ctx->setpriv()
and $ctx->getpriv() is necessary because each callback
function could be called from entirely different threads. These
functions allow you to share data regardless of which thread it
is being called in; Milter simply takes care of this for you. See
the Sendmail::Milter documentation for more information on the functions
that can be used via the $ctx context object and the other
SMFIS_* return options that are available to you.
Conclusion
The Milter interface provides the ability to manipulate messages
with any arbitrary rules you can imagine. It also gives you full
control of the SMTP process at each step so that you can accept,
discard, or reject messages very early in the SMTP transaction.
Milter is also the method used in many commercial filtering tools
such as Sophos' PureMessage, Roaring Penguin's CanIT, and Brightmail's
Anti-Spam software.
I've successfully used filters written in Perl in large-scale
environments without any serious performance problems. It should
be noted, however, that a filter written in C consumes fewer overall
memory resources than one written in Perl. There are also Milter
interfaces for several other languages such as C++, Python, and
Java if you are more comfortable with these. Happy Miltering!
Frank Wiles is the IT Manager for Sunflower Broadband in Lawrence,
Kansas (http://www.sunflower.com). He also does systems
administration and programming consulting via his company Revolution
Systems, LLC. Frank can be reached at: frank@revsys.com. | 
