SPF

SPF (Sender Permitted From) is a new machanism which allows you to define what ip addresses are permitted to send mail 'from' your domain, this will stop spammers from pretending to send message from your domain.

What you should do?

Define an SPF text record in your DNS entry for your domain(s) (this is not required but is a good idea)
If using surgemail, update to SurgeMail 2.0e or later and add these settings:

                g_spf_mode "strict"
                g_spam_block "true"          (optional, or you can turn it on per domain or per user)
                g_spf_rewrite "true"

What setting makes it block messages ?

Nothing is blocked when you set g_spf_mode "strict" so that is always safe. When messages are blocked they are blocked using the SurgeMail 'allow' system so that the sending user (if they are a real human) can fix the problem without intervention from an administrator.

To actually block spam you must do one of these:
1) Set g_spam_block "true" (that sets it on for everyone by default)
2) Set the domain setting spam_block "true" (which sets it on for everyone in a given domain)
3) In the web user settings, login and click on SPF on the left hand side, then set the option box at the top to "True"

Doesn't SPF rely on the senders creating spf records ?

No, in strict mode surgemail makes up an spf record for all incoming domains so it works for everyone. When the made up spf record fails (which is rare) surgemail then provides other checks and mechanisms so real email can still get through.

What about 'DomainKeys' ?

DomainKeys is a cryptographic solution which is similar to SPF, in general, SPF does everything that domainkeys does, but without the extra load and complexity of cryptography. We recommend you also use domainkeys tests (available in the latest versions of surgemail) for incoming email to cope with the two large providers who refuse to define SPF records (apparently for political reasons).

Also see SurgeMail Spam Prevention Guide

Why/How will SPF stop spam?

There are two types of spam, legitimate businesses sending email from real domains to people who haven't asked for it, this type of spam is annoying, but trivial to filter with simple rules and RBL databases. And most businesses are learning not to do this as they rapidly find themselves cut off from the customers they do want to talk to. This type of spam will continue but at a relatively lower level, it isn't really a problem.

The second type of spam is the problem, it's sent by people who use fake 'from' addresses and domains, via multiple ip addresses and virus mail slaves, meaning each email comes from a new ip address, each email is written specifically to evade the filters, and new variants are written each day. These mails are more or less impossible to filter. However, this second set is trivial to block with SPF!!!

SurgeMail and SPF (Cool features)

Full implementation of SPF, SRS and Macro's, all controlled with simple config settings (Install version 2.0 or later)
Built in test page https://localhost:7025/?cmd=spf for testing SPF processing on sample addresses.
Additional 'strict' settings to block spammers while letting through most legit email even if SPF records don't exist.
Additional 'allow' mechanism so false positives can be corrected without administrator intervention
Internal long term IP database of known addresses, known spammers, and known non spammers, linked to the 'aspam' system.

Turning on SPF with SurgeMail

Add the setting 'g_spf_mode', set it to "block", or "stamp", or "strict"

g_spf_mode	Meaning
block	Bounce emails that come from a domain with a valid SPF record if the SPF return code is 'FAIL' By default the 'allow' mechanism will be used so nothing is actually blocked unless you set the domain or user default to 'block'. The domain setting to make it actually block by default is spam_block "true". Note: "block" mode will not block email from domains with no SPF record, use "strict" mode instead.
stamp	Do the SPF processing and add a header to the message, which is then used by the filter to score the message more accurately as spam
strict	This setting is unique to SurgeMail, it means if there is not a good 'SPF' record then SurgeMail applies a default rule "mx/16 a ptr:%{d2} -all" settable with g_spf_default which basically means is the person sending this message from a mail server that accepts messages for this domain (or close to it) or if I lookup the name of this ip number, does it match the domain name. Then SurgeMail checks if it's an ip address it knows about historically, and if still can't find it, then it blocks the message. This will block some real email, but it will block a 'lot' of spam. By default the 'allow' mechanism will be used so nothing is actually blocked unless you set the domain or user default to 'block'. The domain setting to make it actually block by default is spam_block "true" Set strict for ASPAM scoring only If you just set strict mode, but don't set spam_block "true" or g_spam_block "true" then the messages will be stamped only, the aspam rules will notice this and score them accordingly, so it is useful to set strict mode without setting anything to actually block messages.

g_spf_mode

Meaning

block

Bounce emails that come from a domain with a valid SPF record if the SPF return code is 'FAIL' By default the 'allow' mechanism will be used so nothing is actually blocked unless you set the domain or user default to 'block'. The domain setting to make it actually block by default is spam_block "true". Note: "block" mode will not block email from domains with no SPF record, use "strict" mode instead.

stamp

Do the SPF processing and add a header to the message, which is then used by the filter to score the message more accurately as spam

strict

This setting is unique to SurgeMail, it means if there is not a good 'SPF' record then SurgeMail applies a default rule "mx/16 a ptr:%{d2} -all" settable with g_spf_default which basically means is the person sending this message from a mail server that accepts messages for this domain (or close to it) or if I lookup the name of this ip number, does it match the domain name. Then SurgeMail checks if it's an ip address it knows about historically, and if still can't find it, then it blocks the message. This will block some real email, but it will block a 'lot' of spam. By default the 'allow' mechanism will be used so nothing is actually blocked unless you set the domain or user default to 'block'. The domain setting to make it actually block by default is spam_block "true"

Set strict for ASPAM scoring only

If you just set strict mode, but don't set spam_block "true" or g_spam_block "true" then the messages will be stamped only, the aspam rules will notice this and score them accordingly, so it is useful to set strict mode without setting anything to actually block messages.

When SurgeMail does bounce a message (which will be rare anyway due to the other checks) it will give an email address that the sending person can send to to 'allow' their ip address. Then that user will be able to send without problems. The entire net block is enabled x.y.z.* so related mail severs won't get problems.

The following additional settings may be useful too:

g_spf_domain "main.domain.name"

This is the domain spf will use for the unblock message set this if the default domain it chooses is not appropriate. Make sure email addressed to this domain will arrive on this server (e.g. add an mx or a record if needed)

g_spf_rewrite "true"

This enables SPF/SRS rewriting of from addresses on forwarded email, which is crucial if other servers are obeying SPF records, you only need this setting if you allow users to forward email to other servers. If you have multiple incoming mail servers copy the file srs.secret to all of the servers in question (it should contain a random number.

g_spam_allow_rbl "true"

This applies the same 'allow' mechanism to RBL/ORBS lookups, which means it's much safer to use a 'deny' action instead of the softer 'stamp' action.

g_spam_allow_msg "||reason||, send to ||allow|| then resend your message."

This lets you change the format of the 'allow' error.

g_spam_allow_known "true"

This is good to set as it means only new 'unknown' ip addresses are blocked for SPF reasons.
so it's not a transient spammer"

g_spf_skip_from "noreply@*.paypal.com"

Use this setting to selectively allow from addresses from badly behaved systems which routinely come from random ip addresses, like paypal. As long as you keep this list short and fairly specific it will be very difficult for a spammer to make any useful use of it.

g_spf_rev_skip "*.ebay.com"

Setting to skip spf checks if reverse ip matches, e.g.

         g_spf_rev_skip "*.yahoo.com"
         g_spf_rev_skip "*.ebay.com"

Use to define any badly behaved forwarding sites (sites that forward email for
many domains but don't correctly use SRS rewriting rules.

g_spf_share "other.mx.host"

Use this setting so when an 'allow' message is received and is valid it is then forwarded to other mx hosts so the information is shared. You must also copy the srs.secret file to all mx servers.

g_spf_default

This lets you over-ride the default rule which is applied when a domain has no spf rule. The default is:

mx/16 a ptr:%{d2} -all

However the 'd2' is replaced with 'd3' for country level domains. So generally this default is best left alone.

Settings which control what SurgeMail does with an spf failure

Setting	Default	Description
g_spam_block	false	If set, then if spf checks fail message is blocked
g_spam_block_gateway	false	If set, and if message is being gatewayed, then if spf checks fail messages is blocked
spam_block	false	If set, then if message is for this domain, then if spf fails then message is blocked.
spam_block (authent response)	None	This is the authent response, which can be set to one of three values 'not set', 'true', 'false'. 'if not defined' = Message is blocked or not based on global and domain rules above. 'true' = If spf fails, message is blocked ignoring above global and domain rules. 'false' = Messages for this user are never blocked.

Gateways & Filters and secondary MX hosts

SPF must be implemented on the servers that receive email from the outside world, so if you have filters in between the internet and your real mail server, then you need to enable spf on the filter system.

The same applies to secondary MX hosts, but in this case you must do one of two things so that the messages from the secondary mx host are accepted by the primary host, e.g.

With SurgeMail version 2.0h or later use g_spf_skip "x.x.x.x" where 'x.x.x.x' is the ip address of your other mail server(s) (a comma seperated or wild card list).
Turn on g_spf_rewrite "True", which means anything that is accepted for delivery by the secondary mx is from then on immune to spf checks from 'any' server.
On the primary server list the secondary server as a trusted host e.g. 'g_verify_mx_skip x.x.x.x' where 'x.x.x.x' is the ip address of the secondary server.
Use g_spf_share "other.host1,other.host2" on each server so they can share allow information between themselves.

NOTE: Copy the file srs.secret between all cluster/mx servers so it matches on all systems, this file contains a random number which is used to verify srs and allow emails. This file will only exist once you start using SPF, it resides in the g_home directory.

tellmail spf_export file.name
tellmail spf_import file.name

Exports the list of 'known' ip addresses to a file suitable to copy to another system, use this when adding an incoming mx gateway if you want it to already know about all existing 'trusted' ip addresses. The file name is relative to the surgemail home directory, not your current working directory!

Authent Module support

Your authent module must support an extra string field 'spf_block'

spf_block, valid values are 'true,false,blank'
true = block
false = don't block
blank = system default.

+OK user@here.com 0 spf_block="FALSE"

Debugging

See the file allow.log to see why messages are let through or blocked when using these settings.

How to define an SPF record for my domain

Start with something like this, replace 1.2.3.4 with the ip address of your mail server.

Add this as a 'txt' record for you email domain, e.g. example.com
	 v=spf1 ip4:1.2.3.4/24 -all

In addition you should probably add a DMARC record to tell other servers to enforce spf for your domain:

So add the dns entry:  _dmarc.example.com
A  'txt' record containing:     "v=DMARC1; p=reject; aspf=r; adkim=r"
or if you want reports:     "v=DMARC1; p=reject; aspf=r; adkim=r; rua=mailto:dmarc-feedback@example.com"

That's probably all you need. You can send from any address 1.2.3.0-1.2.3.255 which will probably be fine for most purposes. It's simple, and efficient. If you want though you could add a little more. But remember the more you add, the slower it is to process, and the more likely you will make a mistake. Here is a good basic rule that adds in the mx records which will help if you change things in future...

 	 v=spf1 ip4:1.2.3.4/24 a mx -all

Token	Explanation
v=spf1	Version of SPF syntax
ip4:1.2.3.4/24	Allow any ip address 1.2.3.0-1.2.3.255 (change to match your own mail server ip address)
a	Allow any ip which matches the IP address of this domain (doing a simple 'a' lookup)
mx	Allow any ip which matches the IP address of a mail server that accepts incoming mail for this domain.
-all	Block any mail from an ip other than those listed above

How do I check my spf record for my domain

The best way is to use the SPF test page in surgemail (Spam Control - SPF Settings - Test SPF Records). This will allow you to test any sending ip address + from address + spf record combination. This allows you to:

Check your new spf record before configuring your DNS server
Test any domain to see whether an spf record has been setup, and setup correctly

eg sending mail from valid (216.65.3.228) and invalid (1.2.3.4) ip address for surgemail-support account on domain netwinsite.com :

 spf: Start ip=216.65.3.228 from=surgemail-support@netwinsite.com
 spf: lookup (txt) (netwinsite.com) --> (v=spf1 ip4:216.65.3.0/16 ip4:210.54.44.0/16 a mx -all)
 spf: SPF txt record found (v=spf1 ip4:216.65.3.0/16 ip4:210.54.44.0/16 a mx -all)
 spf: +++ Proccessing Token=(v=spf1) Parameter=(spf1) cidr=32 +++
 spf: Processed token (v=spf1) - continuing  .........
 spf: +++ Proccessing Token=(ip4:216.65.3.0/16) Parameter=(216.65.3.0) cidr=16 +++
 spf: spf: cidr(216.65.3.228, 216.65.3.0, /16) Matched Mask
 spf: Last token {ip4:216.65.3.0/16} (res=PASS)


 spf: Start ip=1.2.3.4 from=surgemail-support@netwinsite.com
 spf: lookup (txt) (netwinsite.com) --> (v=spf1 ip4:216.65.3.0/16 ip4:210.54.44.0/16 a mx -all)
 spf: SPF txt record found (v=spf1 ip4:216.65.3.0/16 ip4:210.54.44.0/16 a mx -all)
 spf: +++ Proccessing Token=(v=spf1) Parameter=(spf1) cidr=32 +++
 spf: Processed token (v=spf1) - continuing  .........
...
 spf: spf: cidr(1.2.3.4, 216.65.3.228, /32) No match
 spf: Processed token (mx) - continuing  .........
 spf: +++ Proccessing Token=(-all) Parameter=() cidr=32 +++
 spf: Last token {-all} (res=FAIL)

An alternative way to check your SPF record is using nslookup or dig as per:

 	 [root@linux]# nslookup
 	 > set type=txt
 	 > netwinsite.com
 	   Server:         10.0.0.25
 	   Address:        10.0.0.25#53
 	 Non-authoritative answer:
 	   netwinsite.com text = "v=spf1 ip4:216.65.3.0/16 ip4:210.54.44.0/16 a mx -all"

Other SPF resources: