services statistics projects policies software support resources staff contact us NSG home UofT home

UTORmail: Junk Email (spam) Filtering This documentation has been written for technical support staff, the technically advanced and the curious. Those looking for an overview and basic instructions, please visit www.utoronto.ca/ns/antispam. Here's How It Works -- In Detail Anti-spam filtering is a two step process. Each incoming message is given a "spam score" and then either placed in your Inbox or placed in your junk-mail folder depending on the spam score and the customer set risk level. When messages arrive at the University of Toronto UTORmail postoffice, they are scanned with a system called PureMessage and given a spam score ranging between 0 (probably not spam) and 100 (almost certain to be spam). The spam score is added to the message in a special line in the message header. This line is usually not shown, but with most email programs you can see it if you want to. The message then goes to the email server. If you have spam filtering turned OFF, the message goes to your Inbox. If you have spam filtering turned ON, any message with a sufficiently high spam score (defined by your chosen risk level) is placed in a special email folder named junk-mail. Any messages in junk-mail older than seven days are regularly DELETED.   Identifying Junk Email (spam) PureMessage looks for many message characteristics common to spam including certain phrases or words; inconsistencies or errors in the message header; use of color, bolding, or capitalization; the use of certain routes to deliver the message; whether the message already appears in a large catalogue of known spam, etc. The scoring system is regularly updated by Pure Message to improve its effectiveness and deal with changes in the characteristics of spam over time. Many aspects of the message are considered in coming up with the final spam score. In practice, the scoring system has been found to be quite effective at identifying what most people consider to be spam. Still, just because a message looks like spam to PureMessage does not mean that it is spam. You may choose to receive marketing email messages from a company you purchased a new computer or a book from, for example. Also, some responsible organizations you may want to hear from use the same email distribution programs as spammers or might like formatting their messages in spam-like ways. To be sure that you do not lose messages you want, it is important to regularly check your junk-mail folder. Any message in your junk-mail folder that is more than seven days old will be regularly deleted! Viewing the spam score of a message When PureMessage decides a message looks like spam, it adds a line to the message header. With most email programs, this small change will not be evident when you view the message because they usually show only a few header lines, like Date:, To:, From: and Subject:. If you would like to see the header line containing the spam score, you'll need to have your email client program display the full details of the message headers. How that is done depends on which email client you use. Here is an example of what you might find: X-PMX-Spam: Gauge=XXXXXXIIIIII, Probability=66%, Report="MAILTO_TO_SPAM_ADDR, NO_MX_FOR_FROM, ONLY_COST, RAZOR2_CHECK, SPAM_PHRASE_02_03, SUPERLONG_LINE" The spam score is indicated by the "Probability=66%". The "Report" attribute lists keywords for the message characteristics that determined the spam score. Some characteristics are good (characteristic of messages that are not spam) and some are bad (common to messages people consider to be spam). The score is an expression of the aggregate of all the relevant characteristics. For Technical Support Staff The risk levels work as follows: Risk Spam Score filtered to junk-mail Low 80% Moderate 60% High 40%   For Technical Support Staff or Technically Advanced Customers Client Side Filtering Customers with special filtering needs (e.g. someone who wants all spam with the word "mortgage" to nevertheless go to their Inbox, but doesn't want to see other spam), can turn off server-side filtering, and create their own filter rules for the "X-PMX-Spam" header in the email client software (e.g. Netscape Messenger, Netscape Mail, Pine, etc.) This is called "client-side filtering". Doing this is more complex and less efficient. We expect that only support staff or technically advanced customers will be interested in doing this. To do client side filtering, you need to create a rule that looks for X-PMX-Spam: Gauge=XXXXXX. We recommend you use six "X", which will match a probability of 60% or more. More spam will be caught if you use five "X" (matching 50% or higher), but the rate of "false positives" is then higher. Please refer to the client side filtering page for specific instructions on configuring the following email clients: Supported: Netscape Messenger 4.79, Netscape Mail 7.x, Outlook Express Unsupported: Eudora, Mulberry, Outlook XP Acceptlists Each UTORmail customer can provide and Acceptlist, a list of email addresses that are assumed to never send SPAM. When a message from an address matches an address on the Acceptlist, it is never marked as SPAM. Each individual has an Acceptlist. Acceptlists are managed using UTORwebmail as described in http://www.utoronto.ca/ns/antispam. Even though they are managed from UTORwebmail, Acceptlists are applied for all messages received at the UTORmail postoffice, whether read with Webmail, Thunderbird, Outlook Express, Outlook, or any other email client. Acceptlists also apply when a mailbox is forwarded to another email address. For example, a message with 99% probability of SPAM or over will not be forwarded to @sympatico.ca, @hotmail.com, @gmail.com, @yahoo.com, etc., address--but it will be forwarded if the "From" address matches an email address in the Acceptlist. For those that want to manage their Acceptlist manually - Each entry in the Acceptlist must look like an email address. e.g. yichen@oise.utoronto.ca or antoniagryzbow@sympatico.ca - asterisks are permitted, but we recommend that you use exact email addresses to reduce the chance of getting unintended SPAM. e.g. @ecf.utoronto.ca (matches all addresses at @ecf.utoronto.ca) or @somedomain.com (matches @somedomain.com and @abc.somedomain.com) or @somedomain.* (matches @somedomain.com and @somedomain.edu) The technology limits each customer to 100 entries in the Acceptlist. Acknowledgements We wish to thank the University of Washington and the University of Ohio for sharing their anti-spam documentation. We also appreciate the technical expertise that the Unversity of Washington shared with us. 1999 Original May 5, 2008 updated for inclusion of Accept Lists