SMTP stage spam checks

One of the most powerful ways of stopping spam from entering your mailbox is to stop it from being received in the first place!

The way email is transmitted on the internet is via the SMTP protocol. We apply a number of checks at the SMTP stage to stop spammers.

XBL blocking

RBLs (real-time blackhole lists) are a very common way to detect and stop known spam sending machines. The main problem with RBLs is that they often have "false positives" and block legitimate systems, incorrectly causing email to be bounced back to senders.

To stop that happening, FastMail only uses only one RBL for blocking, the SpamHaus XBL. The XBL is a highly accurate block list that lists the IPs of machines with known trojans and proxies. Independent testing shows that the ZEN RBL (of which XBL is a part) has a high block rate, with basically no false positives.

A number of other RBLs and URIBLs are used during the spam checking phase that occurs after email is received, but these RBLs never outright block email; they're used as part of an overall scoring heuristic along with many other factors to filter spam into your Spam folder.

Address enumeration detection

Address enumeration detection is designed to stop other people trying to find what email addresses are valid at FastMail. It does this by detecting attempts to send to many different, but similar, email addresses in a short period of time from a non-email server host.

Address enumeration detection provides a defense against spammers trying to discover valid email addresses simply by trying many different addresses over and over.

RFC violations

When a server is transferring email from one system to another, there's a series of conventions they have to follow defined in the SMTP RFC 2821 specification. Spam bots are often poorly implemented, or try and use tricks to work around these conventions. Many spam bots can be found and blocked by detecting some of the common violations.

Custom spambot detection

Through our extensive, long-term experience running an email service, we've been able to closely analyse how spam bots try to deliver email to our servers. By using this data, we're able to pre-emptively detect spam bots soon after they connect, and block them from delivering their spam email at all.

Greylisting

Greylisting is a process designed to detect and stop spam being accepted from poorly written sending email servers, which almost all spam bots are. One of the main concerns users have with greylisting is that poor implementations will often delay all email. The FastMail implementation uses a number of tweaks to ensure that legitimate email is delivered with no delay, while spam email is delayed or stopped from delivery entirely.

Greylisting works by returning a temporary failure response (4xx code) to the first attempt to deliver an email, but accepting it on the second attempt. Every proper email server will attempt to redeliver a message after a temporary failure response, while currently almost all spam software does not, thus effectively blocking the emails from the spam software.

To avoid delaying proper email servers, our greylisting implementation performs a number of checks on a connection before applying the greylisting policy:

When combined, these features provide an excellent balance of greylisting hosts which should not be sending email, and allowing those hosts which should be sending email to get their email straight through.

Additionally, to help with the tracking of any problems, once a message passes greylisting and is accepted, a new header is added called X-Spam-greylist. This header tells you how many seconds the email was delayed and whether that host has been whitelisted for 24 hours. (Technically, the delay figure actually indicates the length of the last delay for the ip/sender/recipient combination, so in the case of multiple emails from the same person, to the same person, from the same machine in a short time period, the figure will be a bit messy and hard to calculate.)

If you feel that email is being delayed by greylisting, please check the headers of your email for the X-Spam-greylist header first. If it's not present, then the email was not delayed by greylisting, and it was probably just held up on the sender's side, something beyond our control.

Forwarding and custom domains

The result of all these tests is that most spam is blocked before it even enters our systems. However, we can only run these tests when the email is delivered directly to us, not forwarded on from another mail server. If you use your own domain, we strongly encourage you to host your email directly with us rather than having it forwarded on from your registrar, as this will enable us to be much more effective at filtering out spam.