SmiteCRC - Spam detection system

Overview - What is SmiteCRC

SmiteCRC is to spam what a virus scanner is to viruses. It scans each incoming message and marks it as 'spam' with special headers. As the administrator you can then decide to reject the messages that are 'definitely' spam or more likely just let your users know that these special headers exist, they will then use their email clients filtering rules to store messages that have the 'spam' header in a seperate folder. Which they can quickly scan once a day for 'real' email messages and then delete. In this way users who want spam still get it but the tedious sorting of spam and non spam is made enormously quicker.

How does it work? In brief, it uses two systems. A set of rules that correctly identify most spam messages by content and a database of fuzzy CRCs that let it correctly find messages that other people have reported as spam even if they are slightly different on your system. Together the two systems give a very accurate indication if a message is spam.

Following is a technical description of how it works but if you just want to implement it jump to here:

The SmiteCRC client has four primary functions -

• Perform calculations on incoming messages (These values can be used by SmiteCRC to find duplicates).
• Run a rule filter over the email to look for elements common to spam and assigns a spam weighting.
• Download regular updates from the server for both messages reported as spam and filters.
• Check messages at POP time to establish if they have been reported as spam after delivery.

Use of SmiteCRC output (For Users)

SmiteCRC adds two headers to messages that are of use for content filtering along with a CRC header describing the contents of the message.

    X-Smite-CRC: A$3eHhzQ$EjWgRQ$1VD816z#1q4s9KO$63ZKjj#f12aeF$1lUn0u1#e5Eqn2$r3TqJ7
    X-Smite-Match: 1
    X-SpamDetect: ***: 3.581000 Suspect source,Asks you to click below,A word in all caps repeated on the line

X-Smite-CRC is added to the message when it is accepted for delivered and is a summary of the contents of the message. If the recipient of the message considers the message to be spam and submits it to smitespam.com, this header will be used to find matching messages.

X-Smite-Match indicates a match with a sufficiently high weighting has been found for the message in the SmiteCRC database. This header may be applied during delivery or at POP time. Messages with this header should be considered spam and this header can be used for filtering in e-mail clients. If the recipient believes this is not spam the X-Smite-CRC header should be submitted to smitespam.com as non-spam as some legitimate subscription email may be submitted as spam by users.

X-SpamDetect gives the results of the spam filter. A score above 5 would most likely indicate the message is spam. Users should not filter on the presence of this header but on the contents. To simplify filtering, searching for a score of 4 or more can simply be done by searching for "****" in the X-SpamDetect header.

Using SmiteCRC with SurgeMail

SmiteCRC is distributed with SurgeMail and integrated with the web admin interface. To enable simply set "Enable SmiteSpam" to "True" in the web admin interface - this is the default for new SurgeMail installations. In addition the "Allow users to specify specific spam settings" should be enabled. This gives users access to setting their own tolerance to spam and allowing users to store their suspected spam on the server so that they do not have to download it.

Check SmiteCRC is working in SurgeMail by looking at the status output. Near the top, you should see a line similar to the following -

    Smite found 840, late 482, rule file 4224, total 5850

In the above example, SmiteCRC has been running for a while, and the statistics indicate the number of messages that have been scanned and matched.

Lastly you can configure SpamBot if required.

Configuration options