ExchangeDefender v3 UI Update

ExchangeDefender v3 UI Update got rolled earlier this week and we’re still working out some of the issues with the layout and the way information is presented. Overall the system is stable and we’re working on the documentation side of things at the moment. I believe we’re just a few days away from a public availability so please stay tuned.

Own Web Now Forums

We’re quite happy to announce the launch of Own Web Now forums. With the launch of ExchangeDefender v3 we wanted to form a closer relationship with our partners and clients – depending on how folks take to the forums we will expand them to provide support and discussion of our other products.

If you have a suggestion for features to be included with ExchangeDefender v3 please sign up for the forums:

http://www.ownwebnow.com/forum

If it would make it easier to follow via RSS you can subscribe to it here.

Please disable IMF

We have been fielding numerous reports from customers complaining that their email has simply disappeared over the last few days. Either mail sent directly to them, or quarantine reports or anything suspicious.

Over the past two days we have been able to identify two causes of this on target Windows 2003 servers running Trend Micro or IMF. As you may be aware, Exchange team had a brief pause in IMF updates (few weeks) and the most recent update that was pushed out is far more aggressive than the previous ones. It is likely that if you do have IMF installed it is either archiving or deleting messages. In Trend’s case, messages are simply disappearing.

To identify if this issue is affecting you please turn on message tracking and try to search for messages you know you should have received. You will likely see “Message sent to categorizer” at which point there is no further information on the message. With IMF you will see a report by IMF indicating that the message was either deleted or archived by IMF.

These issues are totally unrelated to ExchangeDefender but they will impact the performance of clients that use both for protection. As always, please uninstall/remove/disable mail hygene products to troubleshoot the core of the problem. We have facilities to help you find out if ExchangeDefender is at fault, if you suspect it please call (877) 546–0316

ExchangeDefender v3 Announcement

Dear ExchangeDefender User, Partner, Friend:

Today marks the completion of all the changes and improvements we have been making to the core infrastructure of ExchangeDefender. Thank you so much for putting up with us over the past three months as we’ve virtually rewritten what ExchangeDefender is and how it works. Over the next 30 days we will be enabling all these features will become manageable – we need your feedback. Your patience and your loyalty will not go without notice, if you scroll down you’ll see we’re giving a very nice present to all of our customers and partners.

The purpose of this email is to inform you of the changes to ExchangeDefender and how they will be implemented over the next 30 days. These improvements fit into four categories and were made 100% based on user  and partner feedback: user experience, administrative functionality, centralized (MSP) control and core infrastructure.

User Experience

The user experience of ExchangeDefender will be changing only slightly because we do recognize how sensitive non-IT users are to interface changes. The behavior, location and use of ExchangeDefender will not change to the point that they will require re-training.

– Whitelists, policies and trusts are now implemented in realtime via SQL

– Ability to reassign quarantine management to another user

– Ability to request a resend of daily report

– Ability to search the quarantine

– Ability to mark reviewed messages so only unread counts show

– Ability to receive multiple daily reports

– Ability to adjust the time zone so the times in quarantines and reports reflect users local region

– Ability to mass-release multiple messages from the quarantine

– Ability to view messages over the past 7 days in the Live Quarantine even if the Exchange server is down

– Paging support to limit the number of messages shown on the screen

Administrative Functionality

Administrative functionality will allow the domain administrator to designate a “SPAM Czar” who can be in charge of reviewing the organizations quarantines. Administrator will also have access to make all the changes necessary in realtime. Additionally, the administrator will be:

– Able to add users from the ExchangeDefender control panel

– Resend welcome/activation messages

– Reset passwords, policies, settings for a particular user

– Access all quarantines, messages, archives for a particular user

– Test delivery with internal, external and remote facilities to help troubleshoot mail delivery times

– Ability to establish file level attachment policy settings (still no .exe’s sorry; but you can rename them)

– Statistical breakdown of daily attack profiles and SPAM counts

– Ability to audit outgoing mail and enforce outbound policies

Centralized (MSP) Control

Perhaps the most demanded feature set by a long shot. ExchangeDefender was not originally designed to be reseller friendly, it was designed to protect users from exploits and time consuming SPAM on our own servers. As the product has grown so has its reseller base and we’re providing the management interface you’ve asked us for:

– Ability to self-provision accounts by providing domain, ip, policies and user dumps

– Ability to consolidate entire ExchangeDefender network management under one screen

– Ability to receive realtime billing, user count and statistical breakdowns

– Ability to access any information available for every other screen (user & administrator)

– Ability to provide executive reports

– Ability to retrieve data in realtime for internal reporting systems (web service for Kaseya, LPI, Connectwise integration)

– Branding support for the portal, daily report emails and activation messages.

Core Infrastructure & Training

The most significant changes to the core infrastructure is the way we manage, deploy and support the network. In the past ExchangeDefender nodes had local configuration databases that were generated on-the-fly and loaded at preset intervals. The new ExchangeDefender v3 setup is directly driven by a SQL database and all policies, settings and services are driven by a distributed database.

The most significant improvement is the addition of Live Archive. The live archive gives your users the ability to access their email through a web based interface hosted at Own Web Now Corp – even while the target mail server may be down. This radically transforms ExchangeDefender from a protection system to a business continuity tool. The messages are spooled both on the ExchangeDefender node and a Live Archive system. This way if the server goes down your users will be able to access mail through a secure (and highly auditable) webmail client and the original message will still sit in the queue waiting for delivery to the target mail server. When that mail server does come up the messages will be delivered but your users would have had the option of continuing to write, read and respond to email while you were down preserving company identity and communication channel.

The best part of this is that even though the pricing for ExchangeDefender will be changing, our current active partners will get this feature free of charge and will be grandfathered at the current pricing model even for the future clients that are added to the service.

Changes

All of these changes are influenced by our partners and end users. As such, we want to make sure that we have taken your feedback in account properly and that the changes are implemented in a manner that is most beneficial to our users. As such, we will be deploying one feature at a time, allowing you to use it for a few days, taking feedback and adjusting the behavior and look of the system for everyone’s benefit. Later tonight you will receive an invitation to participate in the design of ExchangeDefender v3, we welcome your feedback. We expect the entire deployment to be complete in less than 30 days as 99% of the code is already written, we just want to make sure it does what you need it to do.

Additionally, we will be investing in the training resources for ExchangeDefender. While half of the security can easily be done by ExchangeDefender alone, the other half consists of properly training the user and the administrator. We will be releasing series of white papers and web video lessons that will help people get started with ExchangeDefender. This will help free you time and make ExchangeDefender deployments less stressful. 

In closing, ExchangeDefender v3 is a drastically different product from v2. It is perhaps the most ambitious effort to provide SMB with a reliable and distributed communication system that leaves the end user in control while providing the security blanket of a large network. I thank you for all your patience with us, thank you for all your money, and hope we continue to deliver the services you’ve come to expect from us.

Sincerely,

Vlad Mazek, MCSE

Changes to Whitelist and Blacklist Functionality

Yesterday (~10PM GMT) we rolled out our new whitelist and blacklist functionality. The new functions allow ExchangeDefender to apply whitelist rules in realtime and minimize the chances of a false positive.

While the entire ExchangeDefender network is SQL driven much of the functionality was not transaction based – we simply regenerated node configuration via SQL which allowed for quick provisioning and central management. Unfortunately, with as large and as diverse of a network as we operate, that also introduced latency in rule updates and gave us a hard time troubleshooting.

So what’s changed? Well, every time you send an email somewhere it is temporarily stored in a SQL database as a reverse whitelist entry. What this means is that we automatically whitelist the people you send mail to. When they respond, they will get through without being put through the extensive SPAM filtering process ExchangeDefender uses. And because the process is SQL driven that recipient is “trusted” in near realtime.

I hope you like this feature.

Note: It took a better part of overnight to switch over to the new process. As result, some previously trusted messages (ie: Yahoo Groups) that are on several blacklists would have ended up in your SPAM quarantine. Our global trusts for our partners (Yahoo, AOL, MSN, Hotmail) were programmed in overnight so all should be well now. If its not, please email support@ownwebnow.com and we’ll address it immediately.

ExchangeDefender Adds Two New Data Centers

ExchangeDefender has enjoyed a very good 2006. As a result of such phenomenal growth, we’re pleased to announce that we are adding two more data centers to our network. We are adding two network locations in Los Angeles, CA and Newark, NJ. These facilities are expected to nearly double the size of the current network by the end of 2007.

If you are currently restricting network access on port 25 to Own Web Now Corp IP address ranges you will have to add two more blocks to the allow list:

64.182.164.0/24 (NJ)
64.182.133.0/24 (CA)

These network ranges are Class C address spaces so you can expect a connection from any host, 1-255. We will start relaying from these locations on Monday, January 15th, 2007.

Adjustments to Tar-pitting for ExchangeDefender

Earlier today we have made some significant changes to the tar-pitting mechanism under ExchangeDefender. The new mechanism is designed to reject messages from hosts that do not follow the proper RFC SMTP dialog and attempt to smash tar-pitting. More on the basic concept of tar-pitting is described here.

The Problem

While tar-pitting is great for throttling remote mail servers and reducing their ability to efficiently deliver a lot of messages, the concept only applies against botnet servers that are attempting to deliver mail in bulk. Anotherwords, tar-pitting is only effective against servers that are concerned about getting the message out as fast as possible. By delaying the SMTP greeting banner, in theory, the remote mail server would have to wait a pre-determined amount of time before starting to send mail. Many open connections at once would overload a single node.

However, spammers no longer exclusively use single nodes in a full force attack. They use the botnet concept by load balancing their broadcasts through multiple servers. As such, those servers connect every few minutes and only relay a single message. By doing so its hard to blacklist them immediately because their overall reputation does not have enough data to be determined. These botnets are designed to bypass tar-pitting by opening a connection and sending data as soon as the connection is opened.

The conversation looks somewhat like this:

Trying xx.xx.xx.xx.exchangedefender.com.

Escape character is ‘^]’.
ehlo spamming-idiot.org
mail from: spammer@spammer.org
rcpt to: vlad@ownwebnow.com
data
Subject: Get a college diploma.
Ohio State University may be a loser but they’ll give you a Ph.D in nuclear physics based on your life experience.
.

Now the (target) tar-pitting mail server has accepted the connection but it has never sent the SMTP greeting. However, it will process the message as soon as its tar-pitting interval passes, thus in part bypassing the tar-pitting and delivering the message. Not good.

Notice that the client above did not wait for the 200 greeting banner, did not wait for the 250 Hello, did not wait for the server to acknowledge the recipient and the sender. They just wrote to the socket and waited. Now even though this does diminish the spammers performance a little (by taking 5 seconds to deliver the message) the message still gets delivered. That’s a problem.

The Solution

The solution is fairly simple: Drop connections with mail servers that are not adhering to RFC. The second the mail server issues a command before the 2.2.0 hostname greeting banner it will get dropped, logged and its tar-pitting interval extended.

Instead of a tar-pitting process that delays the connection a few seconds, this process allows for a connection immediately but delays the SMTP greeting banner a few seconds. As such, it can eliminate server load caused by spammers that think they have found a way around tar-pitting.

We ran this in testing on our production systems and have found 0 false positives over the course of one week. All hosts that were rejected were also on multiple RBLs. The implementation is transparent to the user and administrator and introduces a random (less than 5) second pause on all connections that do not have a reputation rating with ExchangeDefender. Less spam, less stuff to review, less bandwidth and less stress for you.

The Grinch that stole Quarantine Reports

No boys and girls, The Grinch didn’t steal the Christmas Quarantine Reports from Santa ExchangeDefender. They were just delayed while we ran some final tests on the new reporting engine. All OK.

The quarantine reports were sent out at 5:00 PM GMT on Monday, December 25th, 2006. Tonight they will resume their regularly scheduled reporting time of 8:00 AM GMT. The new reporting engine has already improved the performance of ExchangeDefender and opens up possibilities to integrate many features you have been asking for in 2006. We will soon publish our product roadmap for Q1/2007 and welcome input and suggestions from all of you.

Now back to killing SPAM.

ExchangeDefender Report Management Updates

For Christmas this year Santa (Pablo Averbuj) Claus will be bringing our customers a new report system management update. We expect to start on Christmas eve, December 24th at 6 PM GMT and be done by midnight.

There will be no downtime associated with this upgrade and no mail will be delayed. The upgrade will however make some sections of the system inaccessible while your particular domain is being upgraded. Because the upgrade requires us to migrate large collections of data from one database to another we do not want to give you incomplete information during the switch – so if you get your timing just right you might see “We’re moving, come back in 5 minutes” banner when you access your queue reports.

And yes, we’re really doing this on Christmas eve.

ExchangeDefender Begins Managed Tar-Pitting & Back-Scatter

Brilliant title, I know. But give me a chance to explain. Tar pitting and back scatter protection are two technologies that we have been working on for quite some time and in all current tests believe them to be ready for production. First the definitions:

Tar-pitting — The process of selectively delaying a HELO/EHLO response to an open SMTP connection.

Back-scatter — The bounce notifications of messages you never sent to begin with.

Tar-Pitting Protection

Tar pitting is simply a flow control mechanism. By default, SMTP servers answer and handle all connections at full speed. When a remote mail server connects on port 25 the destination mail server responds with its welcome banner and waits for the SMTP dialog to start, eventually leading to message delivery. Because SMTP is unaware of any reputation scoring it accepts communication from anyone in its access list and works as fast as the network and system resources allow. Because all SMTP servers by default work as fast as they possibly can it stands to reason they would become perfect spam targets for spammers that rely on sending as many messages in as little time as possible.

Enter tar-pitting. This mechanism is not new but until now presented significant shortcomings in terms of reputation. For example, your standard tar-pitting mechanisms either had a pre-configured timeout or timed out depending on how many connections the remote server opened. System administrators could configure their own tar-pitting settings which usually ended up in two unforeseen circumstances:

• The server would wildly fork and keep open processes listening on port 25 until all system resources were exhausted.
• The server would not recognize whitelists or reputation – applying the same throttling process to all mail server. So a large AOL mail exchanger that likely delivers thousands of messages a day to your organization would be greeted with the same delay as the SBS server that only delivered a few every other week.

Both approaches were faulty and resulted in a significant mail delay as well as performance degradation.

ExchangeDefender Tar-pitting

ExchangeDefender approach adapts our scoring and reputation technology. Servers become trusted over time and in real-time. Our research indicated that there are two polar types of mail servers out there: massive mail gateways and low volume mail servers. Massive mail gateways can be evaluated and over time trusted or whitelisted to get immediate service and reduce the amount of time they would have to wait for delivery. It stands to reason that a high activity mail server legitimately sends large volumes of mail over time.

However, if a low volume mail server started attempting thousands of connections per minute to our network we would get suspicious. Enter tar-pitting. By throttling low volume mail servers we can reduce overall network load and allow our spam filters to catch up in case these low volume mail servers were compromised to send malware, viruses, trojans or spam.

Back-scatter Protection

Back-scatter is simply the returned message (“bounce”) from a mail server that replied to a message you never sent. This threat has been around since the early days of commercial Internet in form of Joe Job attacks. In a Joe Job attack a spammer would assume identity and forge (“spoof”) a message to a remote recipient. The remote recipient would get upset at the “forged” identity and spammer would be in the clear.

Many faulty spam control mechanisms and Microsoft Outlook / Outlook Express exploits over the years created the environment in which trust was used to implicitly bypass spam filtering. Today when a computer is compromised by a virus or a trojan it can access files on the computer, mailboxes, email messages and more. By scanning the system for email addresses the trojan can build an impressive mailing list and then start spreading itself to those remote systems. The trojan assumes an identity of one of the email addresses on the system and uses it to reach another.

Think of all the email addresses stored on your computer in your address books, text files, accounting packages, etc. Most of them are outdated or gone, some may have their mailboxes overflowing, others have been disabled or have autoresponders on them. Viruses, trojans and other automatically generated content for the sake of spreading around the Internet does not care to validate the existence of the email – it only cares about getting as wide of a distribution as possible.

Thats where back-scatter comes into picture. All those bounced messages can come back into your mailbox to clog it with NDRs or even worse, infected messages that can compromise your security.

ExchangeDefender Back-scatter

Because our mail servers track where you send your messages we can also validate all the back scatter as legitimate or not. Typical non-delivery receipt (“NDR”) lists the subject, time, date, recipient email address and other information that is only known by the sending system and the recipient system. Considering that we control that sending mechanism, we can validate whether the NDR was caused by a message sent from our network or if it is simply a spoof.

Either way, we keep your mailbox clean. 

Implementation

We will begin the wide-spread implementation of tar-pitting and back-scatter protection this week. We have already tested the system adequately and believe that the reduced mail load over the holiday break will allow us to adapt the new processes while the network has more than its usual amount of spare resources.

No change will be required on your mail server and your users and customers will not see a change in the way that mail is being delivered our routed.