Responding to UPS pattern SPAM

It has been quite an evening at ExchangeDefender as we continue to fight the outbreak of the UPS trojan. You may have seen this:

Warning: This message has had one or more attachments removed

Warning: (UPS_INVOICE_978172.exe, UPS_INVOICE_978172.zip).

Warning: Please read the “ExchangeDefender-Attachment-Warning.txt” attachment(s) for more information.

Subject: UPS Tracking Number 6431834482

Unfortunately we were not able to deliver postal package you sent on July the 1st in time because the recipient’s address is not correct.

Please print out the invoice copy attached and collect the package at our office

Your UPS

What is interesting about this is that the message does look fraudulent to the casual observers and people that do domestic business with UPS. However, we have encountered this format (with attachments and all) being used by UPS Commercial shipping departments in the past, which is why messages with the specific patterns received lower SPAM scores and were allowed through.

We still stripped the attachments but the attachments inside the ZIP file are passing through AV scanners as the variants change. We are now up to over thirty definitions used to track this specific worm and have taken the following steps:

UPS messages are only processed if they come from UPS.

UPS Tracking numbers are only accepted as valid if they start with 1Z.

UPS messages instigate a callback function against UPS servers.

Dealing with these extended rulesets and checks has made mail move a little slower today as we’ve dealt with onslaught of messages while this worm becomes more prevalent. UPS is also issuing a warning on their behalf:

We currently have this issue under control and it should not pose any further problems. However, expect the UPS messages to be taken with higher scrutiny and always warn users not to open executable attachments.

ExchangeDefender LiveArchive now integrates outbound mail

It has been quite a while since we introduced a new feature to ExchangeDefender. This is mainly because we have been hard at work on ExchangeDefender 4.0 whose LiveArchive backend is completely different from the present one. But we have heard the pain and thanks to the number of very compelling arguments we now support transparent archiving of outbound mail as well as inbound mail via LiveArchive.

Service is already provisioned and active for everyone that relies on LiveArchive. You do not have to do anything to activate the outbound archiving component. This new feature brings us one step closer to giving you a fully redundant mail solution within ExchangeDefender portfolio.

For more details on LiveArchive please see the ExchangeDefender web site LiveArchive overview.

Stepping up NDR and DSN filtering response

Due to the enormous amount of feedback by our customer base we are stepping up the defense from NDRs received for the emails that were not originated by your users to begin with. This is often called NDR blowback, backscatter, fake virus or worm storm, etc. It happens when someone uses your email address to relay an enormous amount of SPAM to the remote servers and encounters a lot of dead mailboxes that may have already been removed or had their quotas filled with SPAM. Naturally, an error bounces back to you because the remote server thinks you sent it.

We have had NDR backscatter protection for quite some time but the cries from our customer base have forced us to take away our liberal stance on this issue. We are now strictly enforcing NDR legitimacy, meaning that we will only deliver NDR mail if the message was sent through one of our outbound servers. Anything else, because we cannot validate it, will be automatically thrown into the SPAM queue if you choose to quarantine SPAM messages.

Are NDRs SPAM?

No, the non-delivery receipts and delivery status notifications are not SPAM. They do not contain any unsolicited commercial communication, they are not selling anything, they are not dangerous in any way. They are annoying, very annoying when you receive a few hundred in a span of a minute. How did this happen? Well, someone you previously emailed likely got infected by a worm or a virus that searched their hard drive (mailboxes) for email addresses. It then took a random address and joined a botnet and sent thousands of messages and made them appear they came from you. Because the remote (recipient) server did not have proper SPAM protection it blindly accepted the message and issued a rejection.

How does ExchangeDefender know what passed through it and what did not?

ExchangeDefender outbound network stamps each outgoing message with a hash key. When the message is returned in a form of a DSN or NDR we check the SMTP header for the presence of our hash key, we decode it and compare with the local copy stored in our server along with the matching From: message. If the hash key matches the sender of the message the email is passed on to other filters. If it doesn’t it means that  the message is a bounce to the message you never sent in the first place because it did not go through our network and it did not get stamped.

What to do if you still keep on getting NDRs?

There are a few things:

1. Check that you are sending mail using outbound.exchangedefender.com as your organizations smarthost.
2. Check that you only have inbound30.exchangedefender.com as your only MX record. If you have more than one your configuration is broken, follow the deployment guide.
3. Check that you are enforcing IP restrictions, port 25 only and from our exchangedefender.com network only.
4. If everything looks correct and the NDR was received after Tuesday, May 10th, open a support request with the text of the NDR as well as full SMTP headers of the message for review.

Thank you for trusting us with your mail.

Upgrades to ExchangeDefender SPAM Monitor and Shockey Server Agent

The long awaited upgrades to our client software are finally out and available for download below. These updates address the issues found in the original releases that prevented the systems from rebooting in certain circumstances. This is a bugfix release only, if you’re not having problems there is no need to download them.

First up, ExchangeDefender SPAM Monitor that alerts you of SPAM waiting for you on the server:

ExchangeDefender SPAM Monitor

SpamMonitor_Setup_v.1.0.2.exe
SpamMonitor_Setup_v.1.0.2.msi

 

Second, the Shockey Monkey Server Agent software designed for Microsoft Windows (2000, XP, 2003, Vista and 2008) used to collect server inventory, logs, WMI data and intelligently feed it to Shockey Monkey for managed services and asset management:

Shockey Server Agent

ShockeyServer_Setup_v.1.0.2.exe
ShockeyServer_Setup_v.1.0.2.msi

Two builds are provided as .exe and .msi, you only need one. The .msi build is special because it can be used to roll out the software automatically using third party management tools.

When [SPAM] and [SureSPAM] slide through

Lately we have been fielding a lot of questions about why [SPAM] and [SURESPAM] messages keep on sliding through to the end users. We have also seen a lot of activity with users complaining about SPAM making it to them uninterrupted when it comes from an email address within their domain. Here is the problem:

In nearly all cases that we investigated, the user actually whitelisted their own domain or their own domains email address.

Why would this happen? Well, users tend to scan messages and look for familiar names and subjects. When they encounter something they recognize, like an email address from their colleague or from themselves, they trust the sender. When they trust the spoofed address, all future mail comes through, causing frustration for everyone involved.

Advise your users not to trust their own email address space when it shows up in ExchangeDefender SPAM reports. ExchangeDefender only intercepts messages going in and out of the organization, it does not filter internal messages. Any mail with the domains address space caught by ExchangeDefender is highly likely to be spoofed.

Of course, usage and configuration of ExchangeDefender is up to you, we make the product flexible enough to allow you to set your own policies. But blindly trusting entire domains and mirrored trust sets (from exchangedefender.com to exchangedefender.com for example) will only let dangerous items through. Consider tightening up the ship if you are seeing ExchangeDefender starting to slip, our metrics show that our detection rate keeps on going up as both volume and percentage.

As always, thanks for letting us clean your mail.

New ExchangeDefender outbound servers

With ExchangeDefender 4.x infrastructure already in place on the inbound servers, we are moving our focus to the implementation of ExchangeDefender 4.x on the outbound servers. The new systems will go into production this weekend, April 5-6, and will feature new IP address relay servers:

outbound1.exchangedefender.com 65.99.255.236
outbound2.exchangedefender.com 65.99.255.232

This is the same IP range that is currently in use so you should not have to make any modifications or changes to your systems. Everything will just work transparently.

If you are currently using SPF, which we do not recommend, you will have to adjust it to include the new IP addresses. For illustration purposes, here is a look at the possible SPF record:

domain.com = “v=spf1 ip4:65.99.255.236 ip4:65.99.255.232 ip4:65.99.192.8 ip4:65.99.192.91 a:outbound.exchangedefender.com -all”

This record will restrict relaying for domain.com to ExchangeDefender outbound servers only (naturally you should include your own IP as well for non-smarthosted deliveries) but the above should get you started.

We don’t expect any issues as the new system has been in beta testing for a few months with no significant problems. Performance, logging, reporting and enforcement functionality that it delivers are far beyond comparison with the current service so we are really looking forward to it!

The New Soul of ExchangeDefender

At times in software development it becomes easier to make little optimizations to the process so long as the processing power becomes more affordable and available. Then one day you look up from what you’ve built on your monitor and see a wall of lights, indicators, pie charts and trends that use pretty pictures and colors to paint the portrait of the complexity that manage all the “little” issues you didn’t fix along the way. We are happy to report that after working around the clock for weeks we have regained a lot of control and a wall!

The story of ExchangeDefender is pretty simple. In the 90’s, we could not keep our Exchange servers up, fighting with relaying, viruses, hackers and platform instability became a daily frustration. So we figured, “Let’s put a box in front of it to just scan the junk and pass on valid stuff to Exchange” and it worked. The little AMD K5-100 workstation with an IDE hard drive and McAfee virus scanner did enough to keep our Exchange infrastructure running. Fast forward 12 years, the rise of SPAM, the rise of Malware, the pdf spam, the composite jpg spam, the rise of botnets and we have the ExchangeDefender of today, a 2,800 server network spread over 26 data centers and a full shift dedicated to just racking the new hardware and provisioning new nodes as our company and our load grow.

With data center space starting to shrink, power demands growing, power cost growing and the trend of SPAM increases turning vertical, we had a choice: change the way we manage SPAM flow or start breeding hamsters to generate power. Since nobody volunteered to clean the cage, we changed the way content is passed through and into our network.

The Old Way

Messages were accepted and scanned in the order they arrived with no bias shown for RBLs, origin or content. We took the message, checked it, and if it was clean we delivered it. With the SPAM loads constantly over the 95% range, legitimate messages had a 95% chance of being scanned with less urgency than their SPAM counterparts. To further complicate the issue, we scanned all messages equally: SPAM checks, heuristics, viruses, RBL presence, message integrity, the works. This created an immense demand for processing power, storage and network capacity.

Simply put, imagine a window of one second during which 10 messages were received and scanned sequentially from first to last. If the clean message was message #1, no worries. But if it was #10 in the scanning order it would have to wait for us to check for every minor chance that the nine messages before it had content anywhere from a pharmaceutical ad to banking fraud.

The New Way

We first changed the way we rely on our extensive knowledge of SPAM sources and senders and we created a system in which a bias is placed on messages coming from reputable senders. RBLs and statistical models are not perfect, that is why we will never bounce or delete an incoming messages without 100% certainty. Legitimate organizations can end up on RBLs from time to time, etc. But if we received over a million SPAM messages from a host in China with no reverse DNS, why should we treat the message #1,000,0001 from the same IP address with the same priority as the messages from hosts that constantly deliver legitimate mail that our users want to read? The good news is, we no longer do because messages are prioritized well before they hit the beef of ExchangeDefender.

Over the past month we have implemented a new SPAM filtering traffic shaping process that takes messages from known SPAM sources and puts them into a low priority queue to be scanned with far less of our arsenal than the messages from sources that are not as proven of SPAM heavens. So if you’ve mismanaged your infrastructure enough to end up in SpamCop, SpamHaus, SORBS, have an open relay, no reverse DNS the only thing we need to know is if the recipient trusts you. If they don’t, store/drop/delete, if they do, we scan for viruses, complex checks, custom filters and more.

This system allows us to enhance our scrubbing of unknown mail and keep you safe from more innovative threats while keeping SPAM stored just in case. Messages will still be accessible, still scanned. This new process shifts the burden of mail server management on the sender to keep their infrastructure in check while improving the security of our recipient customer. The era of trust on the Internet is long gone, mail servers that are mismanaged to the extent that they spew millions of SPAM messages, viruses and malware cannot be trusted not to have their logs scanned and malware routes replicated through the social means of replaying legitimate mail traffic to bypass filtering systems.

The Story So Far

So far, we have been able to improve nearly all tracked metrics of ExchangeDefender.

The level of SPAM detected is way up. The level of false positives (messages ExchangeDefender thought were SPAM but were released from quarantines by end users) is way, way down. The latency in delivery, scanning and processing are way down. Nearly everything we do has significantly improved over the past few weeks and while it took an enormous amount of hard work we feel it leaves the Internet a safer place for our clients and a far less profitable or even viable business for spammers.

Results

We have done our way to make sure the messages you receive get to you faster, safer while reducing the time needed to manage whitelists, review junk folders or worry about non-receipts. Our support has reflected this as well, despite February being the strongest sales month on record for ExchangeDefender, ever, we have had less support requests, ever!

Now, after a little deserved break, we turn to ExchangeDefender 4.x and think of ways we can use the new savings to make our service work for you even better.

As always, thank you for your business and thank you for trusting us to keep your inbox clean.

Update on ExchangeDefender Maintenance

Generally we keep ExchangeDefender maintenance on the NOC site but due to the extent of the maintenance done we felt it was important to place it here as well. Over the past 12 hours we have conducted urgent maintenance on the following systems:

1. ExchangeDefender reporting servers (ongoing, 8-10 hours remaining)
2. ExchangeDefender antispam engine (ongoing, up to 99.2% efficiency)
3. ExchangeDefender outbound network (complete)
4. ExchangeDefender global load balancing (complete)
5. ExchangeDefender cluster upgrades (complete)

It will take another 2-3 hours for the latency across the network to go down. The email reports will resume later today. The new outbound network hardware is online and active. The new reporting servers will go online this weekend along with the global network upgrade to Microsoft Exchange 2007 Service Pack 1.

We apologize for the inconvenience the additional mail latency has caused you and your clients, this will be the last major network upgrade for quite some time and the last major maintenance before the start of the ExchangeDefender 4.0. Most of the engine underneath of ExchangeDefender is running 4.x code already.

For updates on the ExchangeDefender maintenance please keep an eye on our NOC site at www.ownwebnow.com/noc