Mail filtering

My mail filter is described on this page, in hope that the information might be of use to someone. This filter should work on unix systems which have procmail installed. You can probably make it work also in other environments, such as Windows, but don't ask me how. Other Procmail spam filters can be found from Procmail FAQ link page. Most of them are more extensive than mine. For more general advice on mail filtering please read Spam.Abuse.net's page and Era Eriksson's Procmail FAQ.

Do you need it?

You can use the filter to put mail to separate folders if you receive high-volume mailing lists. Or if you receive more than, say, 50-100 emails per day, then the mail filter might be useful in separating your personal mail, work stuff and spam.

You should have some understanding of using unix and computers in general to use the mail filter described in this page. Setting up and maintaining the filter requires a little work. You do not really need any of this if you don't receive high-volume mailing lists or 100 emails per day -- unless you want to take it in part as a hobby.

Philosophy

I like keeping things simple. Autoresponders are not simple. It is easy to make a working autoresponder, but making a safe one that never gets into mail loops and which cannot be misused is very difficult. Autoresponders are best left for message transfer agents, such as sendmail.

I do not want to attempt to classify spam accurately; use Spam Bouncer or like if you want to do that. In any case, that is something that your internet service provider can do more effectively and reliably (hint: MAPS).

When I send mail, the e-mail addresses of the recipients are automatically appended to my whitelist. Those in the whitelist bypass all spam filtering. I separate mail from mailing lists into separate folders. I classify the mail that is not sent to a recognizable email address of mine as bulk (most of the spam falls into this category).

Installing the filter

Important: Change the configuration variables and email addresses in the scripts to match your system setup and preferences.

Receiving mail

My .procmailrc-file should be self-documenting. Check its settings and copy it to your home directory. You must modify your .forward file to activate the filtering.

I use Mutt mail user agent. You can specify all folders to which procmail delivers mail in Mutt configuration file. For example mailboxes ! =my-list in .muttrc file would mean that you receive mail to the standard inbox (!) and to folder my-list in your own Mail directory. Mutt notifies you if there is new mail in any of these folders as it would do for the standard inbox.

Mutt highlights the message depending on how procmail classifies it. To this end my Mutt configuration file contains the following lines:

color index white black "~h ^X-Sorted:[[:space:]]Default$"
color index brightwhite black  "~h ^X-Sorted:[[:space:]]Whitelist$"
color index green black "~h ^X-Sorted:[[:space:]]Admin$"
color index red black "~h ^X-Sorted:[[:space:]]Blocked$"
color index cyan black "~h ^X-Sorted:[[:space:]]Bulk$"
color index yellow black "~h ^X-Sorted:[[:space:]]List"

Sending mail

I use the procmail recipe file send.rc to tell procmail how to send mail. It appends the e-mail addresses of the recipients to the whitelist and sends the mail using sendmail. You need programs called add-whitelist (a shell script) and fetch-to-cc-bcc (download fetch-to-cc-bcc.tar.gz) which separates email addresses from To, Cc and Bcc fields (I stole most of the code from GNU Mutt mail user agent). Unpack the tar-file and compile fetch-to-cc-bcc with command make.

To use the send recipe you must tell the mail user agent about it. For example in Mutt this can be done by configuration command set sendmail="$HOME/bin/procmail -m $HOME/.procmail/send.rc -oi -oem".

Periodic jobs

You should follow the logs that procmail produces to catch potential problems. Easiest way to do that is to send summary of the logs to you periodically. I do this by having the following line in my crontab file (added with command crontab -e): 20 2 * * 1,3,5 $HOME/bin/mailstat $HOME/Mail/LOG | $HOME/bin/mutt -s 'PROCMAIL LOGFILE' puolamak@pcu.helsinki.fi > /dev/null

Mailstat-script should come with your procmail distribution.

I put send-log to /dev/null: 31 4 * * 1,3,5 $HOME/bin/mailstat $HOME/Mail/LOG.send > /dev/null

The backup directory is also best cleaned using crontab: 30 4 * * * cd $HOME/Mail/backup && rm -f dummy `ls -t msg.* | sed -e 1,100d`

Summary: What to do

As a summary, you must do the following:

Change the configuration variables and email addresses in all scripts to match your system setup and preferences.
Put .procmailrc to your home directory.
Modify your .forward file and test that everything works as you think it should.
If you want to update your whitelist automatically:
- Configure your mail user agent to use procmail to send mail.
- Use procmail script send.rc in sending mail (needs add-whitelist and fetch-to-cc-bcc.tar.gz).
Set up periodic jobs to create log files and to delete old files in your backup directory.

URL: http://www.iki.fi/kaip/mail/filter.html