Enhanced SPAM filtering engine now online

Over the weekend we deployed a few enhancements to the antispam engine and hopefully starting tomorrow you should see far less junk in your Inbox. We have been testing this engine in parallel and have not seen a significant increase in false positives so we are positive this will be an improvement in overall performance and filtering rates.

As usual, if anything slips please forward it to spam@ownwebnow.com

Note on the forwards:

In the event that an obvious SPAM message gets through ExchangeDefender and into your mailbox, you can forward it to us at spam@ownwebnow.com for further inspection. There are times that the SPAM you received was from a source that is not known for spamming or that the message just did not contain enough obvious junk content to be filtered through. We also ask that you forward inline SMTP headers (in Outlook right click on the message in the listing, select Message Options and copy Internet headers) to us so we can verify that the message indeed passed through our network and that it was not released by one of your coworkers that it was cc’ed to (this happens all the time, someone releases the message from a familiar looking sender/subject and its a SPAM that was either cc’ed or bcc’ed to everyone in the organization). For 4.0, we are working on an Outlook addin.

If you are one of our partners and would like to hide the ownwebnow.com domain name, you can have messages sent to spam@exchangedefender.com. If you would rather give your customers your own address just create a mail enabled contact in your organization and forward it to either alias. Be advised that we may sometimes reply from that address in which case they will see our company name and our contact information – however, this is very infrequent! (mostly when people forward a few dozen identical items to which they have obviously subscribed to, such as popular newsletters, shopping sites, etc)

Change of policy in ExchangeDefender non-deliveries

As mentioned previously, we are starting to tighten up the network backend before the UI overhaul of ExchangeDefender 4.0 takes place. In preparing for it we are trying to adjust our network to help reduce load as well as improve the quality of service and transparency.

First the problem: Message sent to user XYZ was not received, can you please check?

There are three possible outcomes to the question above:

1. Almost always the message was delivered properly but the remote users server either discarded it as SPAM or the user outright deleted it and just did not want to fess up to it. We have invested thousands of man hours with our partners where the message from customer A went to customer B and can be backed up with transaction logs of the sending server, ExchangeDefender and recipients mail server. It made it to the mailbox. Then it disappeared. That we can’t do much about but with LiveArchive we have made it possible for our own customers to retrieve those mysteriously undeliverable items.

2. The message was delivered but discarded at some point in the process (ExchangeDefender, third party mailer, broken DNS, etc)

3. Something that we aim to fix: Message was delivered to ExchangeDefender but did not get delivered to the remote server because the remote server uses greylisting, callbacks, long delays, internal content blacklists, guy in the basement writing SpamAssassin rules.

The Solution: Less time in the queue, faster alerts to the user that there is a problem.

Our current settings will retry to deliver the outgoing message for up to five days. Error reports are sent within 24 hours. That is clearly too long as the users complain of issues within hours of sending the email. We will be generating errors sooner than later to help the IT staff and IT Solution Provider explain to the customer that the issue is on the remote recipients server, not on their own or ExchangeDefender which will help with the endless troubleshooting requests over the proven infrastructure just to find out that the recipients server has a poorly configured mail server.

This change in policy will only affect outbound mail, inbound mail will remain the same.

Going forward we will be issuing non-delivery alerts within 3 hours. We will return the message to the sender if the message is not delivered within 24 hours. Based upon the advice of our IT Solution partners and our internal testing, we have concluded that this is an adequate amount of time to attempt to deliver the message and if the delivery fails this gives the sender an option of contacting the recipient through other means until their mail problems have been solved.

ExchangeDefender 4.0 will feature full access to the ExchangeDefender transaction logs both inbound and outbound but in the meantime we felt it was important to introduce this change due to the growing number of hosts that are without protection and just unable to properly handle the amount of inbound mail sent to them. Each undeliverable and unreported issue with the remote server unfortunately increases the cost of support and reduces the satisfaction with the mail delivery because users tend to start blaming their own IT staff first. We hope these errors help the users see where the issue is sooner than later so they can continue to communicate with the remote party through other means.

New policy on ExchangeDefender activations and billing

We are implementing two new policies and processes for ExchangeDefender activations and billing. Some of these have been with the service for a while but since we didn’t enforce them we have opened ourselves and our partners to abuse.

One of our long standing policies is to have a 1-to-1 mapping of all valid e-mail addresses. Every e-mail address on your server that receives email must be on ExchangeDefender as well. This allows us to lock down your server to valid recipients only and thereby reduce a lot of traffic that really shouldn’t exist in the first place.

Going forward, all new ExchangeDefender accounts will be locked down to valid recipients only. Messages sent to non-existent accounts will be rejected with the following error:  “550 ExchangeDefender does not protect this email address (directory harvesting attack rejected)”. If a valid recipient notices this error, make sure you add their account to ExchangeDefender.

Every 20th of the month a new accounts summary will be sent to you so you can adjust your billing for the new accounts that have been created that month. This will allow us all to be in sync with the accounting.

New domains will have their activations provisioned and created within four hours (previously 24). If you choose to do your activations yourself please keep in mind that your ExchangeDefender deployment will be locked down immediately, so do not point the MX record until you have created your users accounts.

In ExchangeDefender 4.0 slated for 2nd Q of 2008, the activations will be immediate and we are also working on a server agent that will keep the server in sync with the ExchangeDefender network eliminating double data entry.

ExchangeDefender 4.0: Scalable LiveArchive

To say that LiveArchive has been a minor part of ExchangeDefender would be a tremendous understatement.

So with ExchangeDefender 4.0, we are taking our business continuity one step further.

Currently, LiveArchive is hardcoded to keep only last seven days of your email on a continuous basis. Unfortunately, business disasters do not come in nice friendly intervals, they can be extended, unpredictable, painful.

With ExchangeDefender 4.0 (scheduled for April/May timeframe) we will be providing LiveArchive that can scale up to 30 days, free of charge! While the default will still be 7 days, you can bump that time frame up and down depending on the needs of your client.

Note: The feature set of ExchangeDefender 4.0 is still forming. If you can think of something that would make ExchangeDefender more valuable to your organization I hope you take a moment to mention it to us. ExchangeDefender is built on user feedback.

Do spammers work business hours only?

One of the frequent comments we get from our users is the apparent sudden increase in SPAM between 9 AM and 7 PM. Overnight, hardly anything comes through but during the daylight hours the spammers seem to get going along with the rest of us!

Yes, yes they do. But not in the way you would imagine.

Most SPAM today originates from workstations, office computers, home computers, etc. There are many studies on the Internet that put the level of systems compromised by a worm or a virus at 25%. That means that one in four computers in use is being abused to send junk mail.

When the workers get to the office and power on their systems, they also power on the SPAM amplifiers that hackers have turned those computers into. With more companies going green and mandating computer shutdowns outside of regular business hours, we see a bigger trend in the SPAM activity start and end times.

This also contributes to delays and deferrals during the business hours. Because networks lay practically dormant overnight, as the millions of computers (“spam zombies”) come online, the mail servers are hit with a huge load that is amplified further by all the “opt in, confirm your email” systems and so on. These tend to overload the mail servers and cause huge delays and disconnects all over the place.

This in part is why we only troubleshoot issues during business hours. After hours everything appears to run correctly because there is significantly less load placed on the network. However, those tasks during business hours can quickly identify a host that is overloaded and not taking any mail.

Antispam Engine at 100%

We have been closely monitoring the Antispam engine since the upgrade took place on Tuesday and so far the performance is quite impressive. We are blocking more messages than ever before and the false positive ratio has gone down as well. What that means is that ExchangeDefender classifies far less legitimate messages as SPAM incorrectly once in every 800,000 messages processed.

Our SPAM training will now resume, please accept our apologies if you happened to receive more SPAM than usual over the last few days. ExchangeDefender uses automated processes to classify messages as SPAM or legitimate mail based on hundreds of thousands of rules, sender reputation, message contents and more. No message is ever screened by a human being. We do monitor honeypots (our mailboxes placed on the Internet for the sole purpose of collecting junk mail) and build message filters accordingly, so if you wish to help by anything that ended up in your inbox that looks like SPAM, forward it to spam@ownwebnow.com (if you can also include the headers that would help a lot as well)

We have already built a set of rules for SPAM that has been emerging lately, mostly Regions Bank forgeries, graffiti.net scatter, Treasury Department Circular 230 and the massive amounts of URI SPAM being distributed through Google – google.com, google.co.uk, google.co.tw and more.

“I am still receiving far too much SPAM”

First, “too much SPAM” is a relative term. If your message has been online since 1993 and is on every mailing list available, you are likely receiving thousands and thousands of messages a day. If a few slip through it is simply due to the volume of mail sent to you – SPAM arrives to you before it does to us, there is no rule to filter for it, and the message gets delivered as legitimate. As the SPAM becomes more prevalent, it starts getting blocked by ExchangeDefender. If you would like to help us filter it faster, forward the message to spam@ownwebnow.com

Second, we only guarantee SPAM filtering efficiency of 99.99% if you have IP restrictions in place. If you are accepting messages from anyone that connects to your mail server than you take ExchangeDefender out of the mail flow and we are unable to protect you. Please ask your system administrator to enforce the IP restrictions and other helpful deployment strategies described at ExchangeDefender Support.

Finally, there is a difference between SPAM and the SPAM you subscribed to. If you subscribed to many stock newsletters, mailing lists, etc, we will not consider them for a SPAM rule. These are legitimate messages and you should address the issues with the sender, not with ExchangeDefender. If that is absolutely beyond consideration, contact your system administrator and request that they put the senders domain (Constant Contact, MarketWatch, Yahoo Finance, Wall Street Journal, etc) on your domain blacklist.

“How much SPAM is too much SPAM?”

On average, a small business organization (5-30 people) receives approximately 20-50,000 messages a day. Of that ExchangeDefender automatically discards roughly 70% just on the sender reputation, multiple blacklists, address book exploits, address harvesting, etc. Of the remaining messages, SPAM tends to compromise 10% and SureSPAM traditionally takes between 80 and 90% of messages, depending on whether there are viruses or worms being spread at the moment.

Overall, ExchangeDefender keeps 96-98% of inbound mail from your mailbox. That means that the combination of legitimate mail and SPAM that is falsely classified as legitimate, accounts for 4-2% of all mail. It is normal to receive a few messages a day, for some of the older addresses and generic addresses (info@, sales@, contact@, bob@) it can be towards a dozen. Anything more than that gets investigated. Anything below that, honestly, is within the fault tolerance of the filters. Remember that these are not screened out by a third party or a live human being, and that these cannot be guaranteed in any way because there is no way to predict what is SPAM before it actually gets sent enough times. SPAM filtering and Virus filtering are not the same thing and the protection does not work the same way. For example, if we blindly filtered every “Viagra” reference no legitimate email containing that word would ever pass through. Same for word fragments, say we filtered every instance of “ass” as SPAM? How many words in the English language contain the letters “ass” in them? Pass? Class? 

Thank you for your business, thank you for your patience with the new engine rollout and we hope this blog post helps explain how the SPAM filtering works and how we can all get rid of more SPAM.

Questions regarding rollouts of ExchangeDefender SPAM Monitor

Several questions have come up in our support portal regarding automatic software rollout of ExchangeDefender SPAM Monitor we released earlier today.

The simplest way to roll out ExchangeDefender SPAM Monitor on a managed network is through Active Directory technology called IntelliMirror, standard in Windows XP, Windows Server 2000-2008 and Windows Vista. The following page describes the basics of the Windows Installer service that can be used to roll out the software package. The process of automatically rolling software out is a complex one and is beyond this article, please contact a trained professional if this is the route you choose. We hope you can understand that we cannot support you on the basics of network management from both legal and confidentiality aspects.

There are several ways to automatically configure the ExchangeDefender SPAM Monitor service for your users, as well as update the configuration from a central location. ExchangeDefender SPAM Monitor uses the following registry hive:

HKEY_CURRENT_USER\Software\OWN\ED

There are three string keys: email, password and suspended. Suspended can be set to True or False, in order for the software to function completely the value should be set to False. Software can be automatically configured without users knowledge by including the .reg file with the cab files mentioned above and the appropriate values for the email and password keys.

ExchangeDefender SPAM Monitor 1.0.2

ExchangeDefender SPAM Monitor 1.0.2 is an optional upgrade for the customers that encountered instability issues with the original release. The original software did not handle 404 errors very elegantly, leading the monitor software to crash if the Internet connection was not available or wifi software launched after the monitoring service did. If you are affected by this issue, ExchangeDefender SPAM Monitor 1.0.2 is for you, if the current release is running without problems the other minor bugfixes may not be worth the download (major fix for multiple email accounts):

 

ExchangeDefender SPAM Monitor 1.0.2

 

Note: Due to popular demand we have also released .cab/.msi files as a part of the package for automatic deployment via Active Directory, Kaseya and other popular management tools.

New Antispam Engine Launches on Tuesday

Our new antispam engine will be launching on Tuesday, January 15th. We have been investigating a number of SPAM patterns over the past few weeks and really building up the techology to fight what we believe is becoming the dominant strain of junk that bypasses virtually all SPAM gateways.

We don’t expect any issues with the new engine as it has been under beta test with our larger clients since Jan 1st.