Skip to content

email: Improve app for managing a full mail server setup

Tasks

The following is the implementation plan/TODOs for mail server app in FreedomBox:

  • Postfix: Install postfix
  • Postfix: Open firewall ports for smtp, smtps and smtp-submission
  • Postfix: Handle configuration upgrades. This is already taken care of as /etc/postfix/master.cf and /etc/postfix/main.cf are not installed by package. Instead they are copied from /usr/share once. Confirm this.
  • Postfix: Configure domains. 'postconf mydestination' should contain {domain_name}. There is no need to configure virtual domain map. This is because our users will have same mailbox on all the domains. And every LDAP user is an actual user on the system.
  • Postfix: Configure using LDAP for user accounts. Actual authentication is done by dovecot. 
    postconf smtpd_sasl_auth_enable=yes
postconf smtpd_sasl_type=dovecot
postconf smtpd_sasl_path=private/auth
  • Postfix: Configure TLS. Postfix can read root owned certificates. This is probably because it reads the keys before lowering it's privileges. 
    postconf smtp_tls_security_level=may
postconf smtpd_tls_security_level=may
postconf smtpd_tls_auth_only=yes
postconf smtpd_tls_cert_file=/etc/letsencrypt/{domain}/fullchain.cer
postconf smtpd_tls_key_file=/etc/letsencrypt/{domain}/{domain}.key
  • Postfix: Setup Let's Encrypt renewal hooks. Simply restart/reload postfix when LE cert is renewed.
  • Postfix: Setup local delivery to go to dovecot. Dovecot supports sieve filters (RFC 5228). These filters can be configured using email clients like Roundcube, SoGo and Thunderbird. While delivering the mail to user's mail box, these filters shall be applied. This allows server side mail filtering that works the same across all the clients used by a user. 
    postconf virtual_transport=lmtp:unix:private/dovecot-lmtp
  • Postfix: Setup mail submission. In master.cf (this should be possible with postconf tool, use that instead) 
    submission inet n       -       y       -       -       smtpd
  -o syslog_name=postfix/submission
  -o smtpd_tls_security_level=encrypt
  -o smtpd_client_restrictions=permit_sasl_authenticated,reject
  -o smtpd_relay_restrictions=permit_sasl_authenticated,reject
  • Postfix: Allow user+list@example.com. This is already default in postfix. 
    postconf recipient_delimiter=+
  • Postfix: Setup fail2ban to handle authentication failures for submission port. Create /etc/fail2ban/postfix-sasl.conf. 
    [postfix-sasl]
enabled = true
  • Postfix: Consider setting up fail2ban to handle repeated failures for mail? Create /etc/fail2ban/postfix.conf 
    [postfix]
enabled = true
  • Postfix: Setup basic webmail addresses (RFC2142). In /etc/aliases 
    webmaster: {adminuser}@{domain}
postmaster: {adminuser}@{domain}
hostmaster: {adminuser}@{domain}
    • Consider setting this up for 'abuse', 'noc' and 'security' too.
    • Run newaliases after changes to /etc/aliases
  • Postfix: Provide UI for adding/removing mail aliases. This works for forwarding too.
  • Postfix: Provide UI for configuring postfix as a smart relay. That is send mail to another server (say gmail with an account and password) for actual delivery. This can circumvent problems that come up when ISPs don't allow mail servers or when DNS control is not available. This is suitable for simple cases when FreedomBox simply wants to notify users with forgot password mails and notifications.
  • Postfix: Backup/restore. /etc/postfix for configuration. /var/mail/ for user mails. /etc/aliases for mail aliases. Start/stop postfix service.
  • Dovecot: Install dovecot-pop3d, dovecot-imapd, dovecot-lmtpd, dovecot-ldap, dovecot-managesieved
  • Dovecot: Open firewall ports for pop3s, imaps (don't open pop3, imap)
  • Dovecot: Handle configuration upgrades. This is already handled by the fact that we can create all our files in conf.d directory without touching the shipped configuration.
  • Dovecot: Configure TLS. Dovecot can pickup root owned certificates perhaps because it drops privileges after reading the certificate. SSL is already enabled by default. Create file /etc/dovecot/conf.d/90-freedombox-ssl.conf 
    ssl_cert = </etc/letsencrypt/{domain}/fullchain.cer
ssl_key = </etc/letsencrypt/{domain}/{domain}.key
  • Dovecot: Setup Let's Encrypt renewal hooks. Simply restart/reload postfix when LE cert is renewed.
  • Dovecot: Allow Postfix to perform local delivery using dovecot. In /etc/dovecot/conf.d/90-freedombox-master.conf: 
    service lmtp {
  unix_listener /var/spool/postfix/private/dovecot-lmtp {
    mode = 0600
    user = postfix
    group = postfix
  }
}
  • Dovecot: Allow Postfix to perform it's auth using dovecot. In /etc/dovecot/conf.d/90-freedombox-master.conf: 
    # Postfix smtp-auth
unix_listener /var/spool/postfix/private/auth {
  mode = 0600
  user = postfix
  group = postfix
}
  • Dovecot: Configure using LDAP for user accounts. By default dovecot allows all system users. However, we only want users in the 'mail' group to have email accounts(?). Create a file /etc/dovecot/conf.d/90-freedombox-auth.conf: 
    !include auth-freedombox-ldap.conf.ext
#!include auth-system.conf.ext
    Configure file auth-freedombox-ldap.conf.ext with auth-ldap.conf.ext as template.
  • Dovecot: Consider enabling LOGIN auth mechanism for supporting Outlook Express and Outlook Mail. Investigate if this is only needed by very old clients and consider not doing this. In /etc/dovecot/conf.d/90-freedombox-auth.conf: 
    auth_mechanisms = plain login
  • Dovecot: Set mail location. This is /var/mail/{user} by default. Nothing to do. We don't want to separate mail boxes based on domain name.
  • Dovecot: Define mailboxes (trash and junk). Junk can be used to automatically move spam emails. Trash is to allow roundcube to delete mails. Carefully consider if they are required and their correct path. In /etc/dovecot/conf.d/90-freedombox-mailboxes.conf: 
    mailbox INBOX.Junk {
 auto = subscribe
 special_use = \Junk
}
mailbox INBOX.Trash {
 auto = subscribe
 special_use = \Trash
}
  • Dovecot: Add plugin sieve (server-side mail filtering). In /etc/dovecot/conf.d/90-freedombox-imap.conf: 
    mail_plugins = $mail_plugins imap_sieve
  • Dovecot: Server side full text search with dovecot-solr (or dovecot-lucene). See https://github.com/progmaticltd/homebox .
  • Dovecot: Investigate setting up OAuth authentication via SASL?
  • Dovecot: Setup fail2ban to handle authentication failures for pop3 and imap. Create /etc/fail2ban/dovecot.conf: 
    [dovecot]
enabled = true
  • Dovecot: Backup/restore. /etc/dovecot/conf.d for configuration. Start/stop dovecot service.
  • rspamd: Install rspamd
  • rspamd: Configure Postfix milters to send mails to rspamd. 
    postconf milter_mail_macros="i {auth_type} {auth_authen} {auth_author} {client_addr} {client_name} {mail_addr} {mail_host} {mail_mailer}"
postconf smtpd_milters=inet:127.0.0.1:11332
postconf non_smtpd_milters=inet:127.0.0.1:11332
  • rspamd: Enable Bayes auto learning. In /etc/rspamd/override.d/classifier-bayes.conf: 
    autolearn = true;
  • rspamd: Automatically send spam to Junk folder.
    • In /etc/dovecot/conf.d/95-freedombox-sieve.conf:
     sieve_after = /etc/dovecot/sieve-after
    • mkdir /etc/dovecot/sieve-after
    • In /etc/dovecot/sieve-after/spam-to-folder.sieve: 
      require ["fileinto","mailbox"];
  
if header :contains "X-Spam" "Yes" {
  fileinto :create "INBOX.Junk";
  stop;
}
    • sievec /etc/dovecot/sieve-after/spam-to-folder.sieve
  • rspamd: Learn spam when user moves mail to Junk
    • apt install dovecot-antispam
    • In /etc/dovecot/conf.d/90-freedombox-sieve.conf: 
      sieve_plugins = sieve_imapsieve sieve_extprograms
      In {} 
      # From elsewhere to Junk folder
imapsieve_mailbox1_name = Junk
imapsieve_mailbox1_causes = COPY
imapsieve_mailbox1_before = file:/etc/dovecot/sieve/learn-spam.sieve
    
# From Junk folder to elsewhere
imapsieve_mailbox2_name = *
imapsieve_mailbox2_from = Junk
imapsieve_mailbox2_causes = COPY
imapsieve_mailbox2_before = file:/etc/dovecot/sieve/learn-ham.sieve
    
sieve_pipe_bin_dir = /etc/dovecot/sieve
sieve_global_extensions = +vnd.dovecot.pipe
    • mkdir /etc/dovecot/sieve
    • In /etc/dovecot/sieve/learn-spam.sieve: 
      require ["vnd.dovecot.pipe", "copy", "imapsieve"];
pipe :copy "rspamd-learn-spam.sh";
    • In /etc/dovecot/sieve/learn-ham.sievec 
      require ["vnd.dovecot.pipe", "copy", "imapsieve"];
pipe :copy "rspamd-learn-ham.sh";
    • sievec /etc/dovecot/sieve/learn-spam.sieve
    • sievec /etc/dovecot/sieve/learn-ham.sieve
    • chmod u=rw,go= /etc/dovecot/sieve/learn-{spam,ham}.sieve
    • chown vmail.vmail /etc/dovecot/sieve/learn-{spam,ham}.sieve
    • In /etc/dovecot/sieve/rspamd-learn-spam.sh 
      #!/bin/sh
exec /usr/bin/rspamc learn_spam
    • In /etc/dovecot/sieve/rspamd-learn-ham.sh 
      #!/bin/sh
exec /usr/bin/rspamc learn_ham
    • chmod u=rwx,go= /etc/dovecot/sieve/rspamd-learn-{spam,ham}.sh
    • chown vmail.vmail /etc/dovecot/sieve/rspamd-learn-{spam,ham}.sh
  • rspamd: Consider downloading and setting up pre-built statistics for Bayes algorithm from https://rspamd.com/rspamd_statistics/
  • rspamd: Consider automatically expunging trash and junk folder contents after 30 days. In /etc/dovecot/conf.d/90-freedombox-mailboxes.conf: 
    mailbox Junk {
  special_use = \Junk
  auto = subscribe
  autoexpunge = 30d
}
mailbox Trash {
  special_use = \Trash
  auto = subscribe
  autoexpunge = 30d
}
  • rspamd: Expose rspamd web interface by forwarding /rspamd to http://localhost:11334
    • Setup single sign-on for logging into this interface
    • Only allow admin (and perhaps mail) users to login to the interface
  • rspamd: Backup/restore. /etc/rspamd for configuration /var/lib/rspamd for learning. start/stop rspamd daemon during operations
  • clamav: Install clamav clamav-daemon
  • clamav: Configure rspamd to use clamav. In /etc/rspamd/override.d/antivirus.conf: 
    clamav {
  attachments_only = false;
  symbol = "CLAM_VIRUS";
  type = "clamav";
  action = "reject";
  servers = "/var/run/clamav/clamd.ctl";
}
  • clamav: Setup fleshclam to regularly update antivirus database.
  • clamav: Setup clamav-unofficial-sigs script to get third party database.
  • Good behaviour: MX record for domain in the DNS 
    {domain} IN MX 300 10 {domain}
  • Good behaviour: Configure DKIM.
    • Generate DKIM key using rspamd 
      rspamadm dkim_keygen -d {domain} -s {timestamp}
    • Store private key safely for use by rspamd. Private key created in /var/lib/rspamd/dkim/{domain}.{timestamp}.key. Ownership should be _rspamd:_rspamd for /var/lib/rspamd/dkim and its files
    • Setup the DKIM TXT DNS record 
      {timestamp}._domainkey.discuss.freedombox.org IN TXT {as output by rspamadm dkim_keygen}
    • Setup DKIM map in rspamd to sign
      • Create DKIM key configuration containing path to DKIM map. In /etc/rspamd/local.d/dkim_signing.conf: 
        path = "/var/lib/rspamd/dkim/$domain.$selector.key";
selector_map = "/etc/rspamd/dkim_selectors.map";
allow_username_mismatch = true;
use_esld = false;
      • Create the file with map from domain name to key name. In /etc/rspamd/dkim_selectors.map: 
        {domain} {timestamp}
  • Good behaviour: Configure ARC. After configuring DKIM copy configuration for ARC: 
    cp /etc/rspamd/local.d/dkim_signing.conf /etc/rspamd/local.d/arc.conf
  • Good behaviour: Add SPF records in DNS 
    {domain}.  IN TXT "v=spf1 mx a ~all"
  • Good behaviour: Add DMARC records to ensure that DKIM is mandatory. 
    _dmarc.discuss.freedombox.org. IN TXT 300 v=DMARC1; p=quarantine; rua=mailto:postmaster@{domain}; adkim=s; aspf=s;
  • Good behaviour: Investigate adding a reverse PTR record. This may not be possible for home users.
  • Diagnostics Is A entry for the domain properly setup?
  • Diagnostics Is MX entry for the domain properly setup?
  • Diagnostics Is DKIM properly setup?
  • Diagnostics Is SPF record setup properly?
  • Diagnostics Is DMARC record setup properly?
  • Diagnostics Is PTR record setup properly?
  • Diagnostics Is the current IP address listed in any spam blacklist?
  • Diagnostics Does the ISP allow port 25 to be reachable. How about newer port for smtps (485)?
  • Diagnostics Tool to check if ports are reachable from outside
  • Diagnostics Tool to send a test mail to local address.
  • Diagnostics Are ports open for pop3s, imaps, smtp, smtps, smtp-submission
  • Diagnostics Auto discovery of email services using SRV records
  • Diagnostics Auto discovery of email services using XML DNS records.
  • Roundcube: Install roundcube roundcube-plugins roundcube-plugins-extra php-net-sieve
  • Roundcube: Enable plugin sieve (needs configuration change).
    • In /etc/roundcube/config.inc.php. 
      $config['plugins'] = array(	 	 
  'managesieve',	 	 
);
    • cp /usr/share/roundcube/plugins/managesieve/config.inc.php.dist /etc/roundcube/plugins/managesieve/config.inc.php
  • Roundcube: Enable plugin for changing password (needs configuration change). In /etc/roundcube/config.inc.php 
    $config['plugins'] = array(	 	 
 'password',	 	 
);
    • cp /usr/share/roundcube/plugins/password/config.inc.php.dist /etc/roundcube/plugins/password/config.inc.php 
      $config['password_minimum_length'] = 10;
      Configure accordingly.
  • Roundcube: Set higher session lifetime. In /etc/roundcube/config.inc.php 
    $config['session_lifetime'] = 60;
  • Roundcube: Setup fail2ban such that roundcube login failures result in bans
  • SoGo: Install sogo
  • SoGo: Setup a basic mail client UI
  • SoGo: Setup a calendar server and UI
  • SoGo: Setup an addressbook server and UI
  • SoGo: Setup sieve filters plugin
  • SoGo: Setup ActiveSync
  • SoGo: Setup fail2ban such that SoGo login failures result in bans
  • Auto discovery of email services: SRV record in the DNS for mail service discovery (RFC6186). Setup DNS entries as follows 
    _submission._tcp     SRV 0 1 587 {domain}.
_imaps._tcp    SRV 0 10 993 {domain}.
_pop3s._tcp    SRV 0 30 995 {domain}.
  • Auto discovery of email services: Add autoconfig XML records (https://developer.mozilla.org/en-US/docs/Mozilla/Thunderbird/Autoconfiguration)

Testing notes:

  • Test delivery with mutt -f /var/mail/{user}/Maildir.
  • Test mail submission with swaks. swaks --to={touser}@{domain} --from=noreply@{domain} --auth-user={anyuser} --auth-password={password} --server={domainname}:587 --tlso.
  • Test IMAP with mutt -f imaps://user@server
  • Test spam handling with 
    wget http://spamassassin.apache.org/gtube/gtube.txt
sendmail john@example.org < gtube.txt
  • Test virus handling with 
    wget https://secure.eicar.org/eicar.com
swaks --to {user}@{domain} --attach - --server localhost < eicar.com

Older body

Difficult: necessary MX records aren't decentralized or zero-configuration friendly.

A few instructions:

http://arstechnica.com/information-technology/2014/02/how-to-run-your-own-e-mail-server-with-your-own-domain-part-1/

http://arstechnica.com/information-technology/2014/03/taking-e-mail-back-part-2-arming-your-server-with-postfix-dovecot/

Edited by Benedek Nagy