Mail Server

ROLE mailhost

The mailhost role configures a machine as a mail server. It receives mail from the outside world and delivers to mailboxes accessible via IMAP. We use Postfix for the mail transfer agent (MTA) and Dovecot as the local delivery agent (LDA) and IMAP server. Rspamd is the only milter used, for spam filtering and DKIM signing. The mailnull role is used on other machines, like the backup server, to relay mail through the central server.

All mail is owned by the vmail user and stored in an encrypted directory using maildir format. The mail user database, DKIM keys, and other data are kept in this encrypted location, usually /vault, for privacy.

All accounts are virtual mailboxes described by an SQL database. Manage this database with the mailcfg script. Background on virtual users can be found at postfix.org. Unix accounts are only for system things and no mail is delivered to them.

We use SQLite for the database because it is small, simple, and sufficient, but MySQL, Maria, or Postgres can also work. On Debian, most postfix daemons described in master.cf run in a chroot, but the "local" and "virtual" do not, so we can keep the database in a central location where it is also used by dovecot to authenticate passwords.

On reboot, postfix and dovecot do not start automatically because the encrypted directory must first be unlocked. Log in and run /usr/sbin/mailboot as root to enter the password, mount the cleartext directories, and start the mail processes.

Variables

The cryptdir_pw is the password for the encrypted mail spool.

# password for encrypted mail spool
cryptdir_pw: "{{vault_cryptdir_pw}}"

The mail_domains and mail_users lists give the domains that we should accept mail for and the virtual users and initial passwords for those users.

mail_domains:
  - "{{ domain }}"
  - second.example

mail_users:
  - email: "{{ admin_email }}"
    pw: "{{vault_mbox_admin_pw}}"
  - email: "relay@{{ domain }}"
    pw: "{{vault_relay_pw}}"
  - email: "user1@{{ domain }}"
    pw: "{{vault_mbox_user1_pw}}"
  - email: "user2@second.example"
    pw: "{{vault_mbox_user2_pw}}"

Define your mail aliases with the mail_aliases list of src/dst pairs. We automatically add aliases to the admin_email for a list of system addresses for each domain. These system aliases include 'root', 'abuse', 'postmaster', etc. and is given by mail_local_admin_aliases.

mail_aliases: 
   - { src: "source@domain",         dst: "dest@domain" }

If you want DKIM signing, the dkim_selector, usually following a year month pattern (202001) so that we have a predictable pattern when we rotate them. This enables signing and generates the keys in the dkim_keyroot directory, which defaults to being under the mail spool. Add the DNS TXT records found in the .txt files to the DNS zones, so that people who recieve your mail can verify that you signed it.

When near rotation time, we can set dkim_selector_next to pre-generate the keys, but not use them. This gives us a chance to get the public keys into DNS and let them propagate.

dkim_selector: 202001
dkim_selector_next:

The mail_server_hostname is what the machine identifies itself as and where client machines try to relay their mail. The default value for this is "mail." on the primary domain.

# defaults to mail+primary domain.
# mail_server_hostname: "mail.{{ domain }}"

Mail Clients

The IMAP and SMTP server for clients is mail_server_hostname, usually "mail." plus the primary domain name. We run IMAP over SSL on port 993, and outgoing mail to port 587 with STARTTLS and plain password. The account name is the entire email address (foo@example.com).

incoming server:	mail.example.com
imaps port:		993
connection security:	SSL/TLS
user name:		somebody@second-domain.example
authentication:		plain password

outgoing server:	mail.example.com
smtp port:		587
connection security:	STARTTLS
user name:		somebody@second-domain.example
authentication:		plain password

We publish a Thunderbird autoconfig XML record on the websites for each domain, so configuring a new account with that should be seamless. This is under .well-known/autoconfig/mail/config-v1.1.xml. Other clients may need manual setup. The DNS settings show how to set SRV records, but I'm not sure how many clients use them.

If a client tries to figure out the settings by hitting the server with multiple variations, it may trip a fail2ban rule, preventing further attempts.

You can change user passwords with the mailcfg script, but users can not currently change them remotely. Obviously, this only works if you have a personal connection with your users, but it can cut out a whole class of password-related mischief. Feel free to do something more sophisticated if you need it.

Mailcfg Script

The mailcfg script is a simple command-line tool to manipulate the virtual user database. It is used by the Ansible playbook to initialize and populate the database. We install it to /usr/bin/mailcfg and you must have write access to the mail database file to use it.

Most people either hand-write SQL queries or install elaborate web interfaces like postfixadmin to do this job. I didn't care for either option, so I put together a simple perl script for the common tasks. It uses DBI, so you can tweak the connect statement and SQL as needed for other RDBs.

Run it as "mailcfg <command>". The following commands are available:

 help		Print this message

 addalias <src> <dst>	Adds alias if missing
 adddom <dom>		Add domain if missing
 adduser <email> <pw>	Adds user if missing

 lsalias [<dom>]	List aliases, all or for a domain
 lsdom			List known domain
 lsuser [<dom>]		List users, all or for a domain

 rmalias <src>		Remove alias
 rmalias -id <num>	Remove alias with given id
 rmdom <dom>		Remove domain
 rmuser <email> 	Removes user

 passwd <email> <pw>	Changes user password 
 disable <email>	Marks user as inactive
 enable <email>		Marks user as active

Spam Filtering

We use rspamd to filter spam. This hooks into postfix as a milter and includes specific rules similar to SpamAssasin and Bayesean classification similar to the old Dspam project. It does both quite quickly. Because it is run as a milter, it can reject the very bad spam before it is accepted for processing.

We use the sieve module under dovecot to deliver messages marked as spam to the Junk folder. It trains the Bayes filter on any good message delivered to the inbox. We also add IMAP sieve scripts to retrain the Bayes classifier when mail is dragged into or out of the Junk folder. The sieve scripts are near the mail spool in the location given by the sieve_root variable. You can also add your own personal sieve scripts.

the rspamd web console at http://localhost:11334 has statistics and fine-grain controls. That port is normally blocked and the server only allows connections from localhost. Set up an SSH tunnel to use it. Either do it from the command line or you can add the following to the ssh config file for that machine.

LocalForward 11334 localhost:11334

We keep per-user Bayes training data, because everyone has different opinions on spam. We do everything during milting and rspamd picks the first of multiple recipients for the statistics.

It would be better to disable Bayes during milting, then classify during delivery, when we have one recipient. However, we should carry over the score from milting. I can turn classification on and off in settings.conf, but haven't figured out how to carry over the score. Probably needs to be done in a Lua plugin. For the future.

This would also have the nice property that we only reject on objective criteria found during the milter pass, and can at most quarantine based on the subjective criteria of the Bayes.

Managing the Mail Queue

To inspect and manipulate mail queue, the following commands are helpful. You can list and flush the queue as any user, but to delete you must be root.

List the contents of the mail queue
# mailq

Flush the queue (try to deliver mail)
# postfix flush

Delete everything in the queue
# postsuper -d ALL

Delete only deferred mail, that would otherwise retry later
# postsuper -d ALL deferred

You can also look at a particular message ID in the queue
# postcat -vq XXXXXXXXXX

Why These Packages?

Back when dinosaurs walked the Earth, there was one mailer, its name was Sendmail, and it was configured by the black speech of M4-dor, which I will not utter here. Lucky people had a leased line, but most just dialed in couple of times a day and swapped messages via uucp.

OK, Boomer moment over. I really do not want to set up a Sendmail system. I can, and it still works well, but great options exist that don't predate the fall of Númenor.

As of this writing, the top three MTAs by number of servers are Exim (580k), Postfix (328k), and Sendmail (80k). That says nothing about the volume of mail passing through each server, but it does give an idea of the general level of support.

The default MTA for Debian is exim4, and it is capable. I did some tests setting it up as a null client. Ultimately, I chose Postfix. It is easy to find documentation for all of the use cases that interest me, has ample performance, and reasonable configuration files. It also supports the milter interface introduced by Sendmail, which is very nice for rejecting spam early in the connection.

Dovecot is the customary choice for Local Delivery Agent and IMAP server. It seems like Cyrus and Courier are other possibilities but I have no familiarity with them.

Rspamd is a relative newcomer to the spam fighting arena but has made its name by combining the functions of SpamAssassin, Dspam, and OpenDKIM into one fast package.

The site.yml playbook does not install a webmail, but if you want one, the webmail.yml playbook will install a dockerized Roundcube image on a separate machine.