ExchangeDefender Detours

Expect some delays in mail delivery starting at about 11 AM EST today. We are adding new servers, new switches, new bandwidth and more routes to our data centers and as we scale each network we will need to refresh configuration and shut that particular load balancer down.

The delays will not be significant and should not be uniform. No mail will be dropped or deleted nor will mail “sit” on the network while we are moving it up. We are taking this opportunity of a really light few days before the holiday to further improve our network and expand our offering (something which you will hear about very, very shortly!)

New SPAM Reports hit Inboxes today!

New SPAM reports for daily and intraday activity are already hitting our customers inboxes today. We have taken so much feedback from our customer base on these reports and taken every bit we could to help improve them. Among notable options, SPAM reports are now:

• Brandable – Your background color, your colors, your corporate identity and product name in both From: and every line that otherwise mentions ExchangeDefender
• Flexible schedule – Reports can be delivered at any 30 minute interval and are adjusted for your home time zone as well as your date format options (m/d/y, d/m/y)
• Daily and Intraday – Daily reports outline past 24 hours of quarantined mail, intraday show only mail since the last report.
• Custom Message – Reports can be branded with a custom message: Alert your customers and users about network events, new services or just general announcements.

Every level of ExchangeDefender user (administrator, service provider, end user) can manage their SPAM report settings and administrators and service providers can now override all settings for all users under their control. Some best practices are to remember not to set intraday reports to run earlier in the day than the daily reports. Also keep in mind that it can take a few minutes for the reports to be generated because they are prepared in realtime.

Finally, remember that daily reports are a great user self-management tools but should not be used as the primary SPAM management option. Create a shortcut to ExchangeDefender instead. To do so, right click on the destktop, select new Shortcut, type in

https://admin.exchangedefender.com/login.php?

username=theirusername&password=theirpassword

Change theirusername and theirpassword values and they’ll have realtime, searchable access to their SPAM quarantines. Enjoy!

Alert: In order to adjust for the reports around the world everyone will receive multiple reports today (one at 9AM EST, and one at their scheduled time). Starting tomorrow you will only see the reports scheduled at your preset time.

ExchangeDefender Mail Delays & Non-receipts

I am writing this blog post to address the issue of ExchangeDefender mail receipt delays or mail simply not arriving at all. Nearly three weeks after we have implemented the new networks, and nearly a month after we have notified all our ExchangeDefender customer administrators we are still fighting with the ad-hoc issues related to delayed mail, mail that was not received, mail that was received hours later.

In 100% of the cases the issue was the recipient policy on the target mail server. Please, please, please make sure you have added the following IP address blocks in order to allow our new servers to relay mail to you:

64.182.140.0/24

64.182.139.0/24

If you do not allow those IP address ranges access to your network the system will not bounce the messages. Instead, our intelligent routing system will route messages internally to the server that is able to establish a connection with you. This system, however, was not designed to handle sysadmin apathy but instead to respond to major interruptions in the Internet backbone. If a system is unable to deliver the message directly to the server it reattempts every 15 minutes. After the first hour it sends a broadcast message asking other networks to see if they can establish a route and receive the SMTP banner. If the connection can be established the message is routed to that server and then delivered. By not having the proper IP address restrictions in place you are forcing your inbound mail to be put through our DR scenario which is automatic but time consuming.

Please, either do not use IP restrictions at all or update them properly. For a little more positive note, tune in later tonight when we’ll announce our new email SPAM reports.

ExchangeDefender Network Status Update

I wanted to offer you an interim update on the status of ExchangeDefender network and codebase. As I mentioned on Friday, we have brought the platform back to normal and since then we have not had any even unusual events (sans a few DDoS attacks which are common against large networks). All our data centers are performing well, now at 11:56 AM which is our peak time we are running at 38% network utilization and 61% system utilization meaning we can sustain twice the load without seeing any effects on the network itself.

All the issues that have been reported over the past week or so have been resolved and we have not had any reports of additional problems since. Now, back to features….

ExchangeDefender LiveArchive

ExchangeDefender LiveArchive is easilly the most popular feature we have introduced in years. Simple to understand why, its a part disaster recovery part business continuity solution that doesn’t cost you anything and works without any maintenance whatsoever. The idea is simple: mail servers and Internet connections tend to be unreliable. Hard drives fill up, Internet connections go down, Outlook profiles or Exchange mailboxes/stores become corrupted our dismount, you name it — we’ve seen it. But the solutions for Exchange business continuity and disaster recovery are very expensive, require appliances and still do nothing when those Internet connections go down.

We have an answer in ExchangeDefender LiveArchive: Your mail, in addition to being delivered if the server is up or queued/mailbagged in case it’s down, is also simultaneously delivered to a separate mail system on our network. We keep a realtime storage of past seven days of your inbound mail along with your profile settings “Vlad Mazek <vlad@ownwebnow.com>” along with all your other mail identities. When you can’t connect to your mail server, you’ll be able to connect to ExchangeDefender LiveArchive server just by going to this web site:

https://archive.exchangedefender.com

Username: your email address

Password: your exchangedefender password

Both should be easy to remember and are even easier to enable. To get started with ExchangeDefender LiveArchive you have to enable it. As a Domain Administrator go to Configuration, scroll down to LiveArchive and select Enabled. If you want this new setting to apply to all the users in the domain just select the checkbox that says “Make this the default setting for all existing users.” and you are set. By default, without this box checked, only new accounts created within this domain will have ExchangeDefender LiveArchive enabled so making sure this box is checked is important. As the ExchangeDefender Service Provider you can accomplish the same by going to Management, selecting the domain to manage and clicking on Change Configuration. Same screen, same configuration.

That is all you need to do and all you need to know. It is important to recognize that this is a true live mail system that you can read all incoming mail, respond to it, delete it, forward it or whatever you wish and can do with your Outlook, Notes or Groupwise clients. The site is always on and always has the latest 7 days of email available. Email is delivered in realtime, meaning that you can communicate with people even while your mail server or Internet connections are down, it does not stop delivering to LiveArchive just because it cannot reach your mail server.

Finally, reading and managing mail here does not mean we start bouncing or redirecting mail – the mail will still be delivered to your server  once it or the Internet connection are back online.

We did everything we could to make Exchange LiveArchive an easy deployment and as easy to use as your own mail client. However, trick to properly executing business continuity is in making sure your employees know this exists and training them on how to access it before they need to do it in an emergency so here are a few checks you need to do:

[ x ] Make sure your employees know and remember the address of https://archive.exchangedefender.com

[ x ] Make sure your users know and remember their email address and password to be used with https://archive.exchangedefender.com – for security purposes this should be different than their server password.

[ x ] Make sure your users are aware of the dangers of using insecure Internet connections, computer kiosks and third party equipment that could be keylogging their usernames and passwords.

[ x ] Consult a lawyer for mandatory legalese in case you have to disclose that your infrastructure is temporarily down. Work on the customer service skills so employees properly notify their contacts that response times may be lagging due to an emergency.

There is a lot more to business continuity but I hope that the ExchangeDefender team at least makes it less of a burgen to communicate in an emergency and gives you the time and resources to deal with high priority problems and not fixing the email.

Unusually high amounts of SPAM in reports today

Due to the recent high-end changes in the ExchangeDefender product (v3.1, NDR handling, archving) we had to rebuild the way we currently manage and process whitelists, trusted senders and outbound mail trusts.

Because this was an urgent modification due to the number of mail servers relaying mail without RDNS, we had to make the change during peak hours yesterday. It in turn affected our ability to properly whitelist incoming messages so if anything seemed like spam, it ended up in the SPAM folders. Yesterday’s report (Monday) contained a large amount of messages that would otherwise been delivered. You can request a release of those messages but you do not need to trust those senders as they are already likely in our whitelists.

NDRs – We spoke too soon

The process we implemented to reject mail from mail servers that have no valid reverse DNS over the last few weeks was unfortunately very short lived one. Unfortunately, we’ll start accepting mail from servers with no reverse DNS entries, allowing our customers to accept mail from mail servers that should not exist on the Internet at all.

After a flurry of complaints we had no choice but to shut down the service that handled the rejections. Instead, we will allow them but will automatically assign a score high enough to qualify the host as SPAM. If this is a legitimate sender to whom you have sent mail in the past, or if you have them on your whitelist, they will get by our SPAM scanners and the host will thus have a better reputation which will keep it out of our internal blacklists. As for others, once they hit that treshold we will place them in our internal RBL and reject mail with the standard disclaimer giving them an option to remove themselves from or RBL’s.

This is not what we would have prefered to do but business is business, email is vital to it and when you complain about how much SPAM is out there remember that you amplify it by doing business with customers and partners that do not have a properly deployed and managed IT infrastructure.

Loose e-mail in the cloud.. with diamonds (ExchangeDefender Mail Tracking)

One of the most frustrating calls you can ever receive from the user is the one that begins and ends with the words: “We are constantly, constantly not getting our mail” – and its your task to prove to the user that you are not at fault (by locating the message). Over 99.9996% of the time the messages end up getting lost through server or client deployed antispam software that should be turned off in the first place. But let’s assume you checked that. Let’s assume you disabled Outlook Junk Filters, Let’s assume there is no IMF on the server. Let’s assume there is no SMTP inspection enabled on the firewall. Let’s assume you have no “security” appliances in your way.

What then?

With ExchangeDefender v3.1 we are giving you full access to our logs, down to the SQL level (more on this to follow). Let’s assume that a user tells you they never received a piece of email, you’re sure everything on their end is working perfectly.. how do we find out if ExchangeDefender messed up? Pretty easy, let’s see where the message went.

Login as the Service Provider

Login with your service provider account to our portal at https://admin.exchangedefender.com

Head over to the new tab, Mail Log.

Select the domain name you wish to run the report on. Search Criteria window will drop down and allow you to put in some basic information (such as the senders email address, subject). Everything except the “To:” field is optional, if you leave both the subject and the from address empty you will see a report of all mail received for that user. All searches are also partial and case insensitive, so you can try to narrow it down if the user isn’t sure of what they were expecting.

Hit find and you will be presented with a list of messages that matched the search criteria. These messages include SPAM, SureSPAM, clean messages, infected messages – basically you see everything we processed.

If you have a question about what may have happened with the particular message you can just click on the Details link (hold CTRL down to open in a new window if you have a few messages) and you will get the following screen with message details:

It is important to note a few things here. First, you are seeing the message details because we accepted and processed the message. If we didn’t, there would be nothing to show. If you encounter that situation, find out who the sender is and open up a trouble ticket. We send rejection notices (as per RFC) to all mail we do not process so if there was an error on our side the user would have received the notice.

Second, it is important to understand that this shows ALL mail processed by us – spam, clean, infected. If it is SPAM it will show a score higher than 0.00. If it is infected, it would say that as well. Also, if you see this message it means it was accepted, processed, and delivered to the user (or dispatched to the delivery queue). Anotherwords, fire up Message Tracking on their Exchange system and dig using the message identifiers noted in the Message Details screen above.

We will have SQL access layer completed by the end of next week which will allow you to see the transaction log down to the SMTP connection and remote queue id signs.

Looose e-mail in the cloud… with diamonds… (its what rings in my head when people tell me they didn’t receive their email)

End of an era at ExchangeDefender

Dear Customers and Partners,

I have just signed off on an implementation that in my mind is long overdue. For years ExchangeDefender processed messages from senders that did not have legitimate reverse DNS entries for their mail servers. As a major supporter of small business and a small business ourselves, we understand that at times it is difficult to appropriate enough funds to properly manage a complex IT infrastructure. As such, we have accepted messages from hosts that do not have valid PTR/reverse DNS because we are aware of how many small businesses rely on the unreliable and poorly configured appliances that generate real business email. Everything from copiers down to portable scanners now generates email of some sort, not to mention the many web sites and services.

However, dealing with this problem over the past year has become impossible to sustain. The amount of worms and botnets launched from dialup, dynamic and even missing DNS has been on an exponential rise and we continue to receive a bulk of our mail from mail servers that would not be accepted as a relay anywhere else. While we want to do whatever we can to assure email delivery, it is simply not feasible to ignore the RFCs anymore.

So, starting today, ExchangeDefender will require a valid DNS entry before accepting your messages. The mail will never be rejected but we will defer accepting it until you (or your IT consultant or your ISP) provide a valid PTR record / reverse DNS for your mail server. Should a message come in from a server that does not have valid reverse DNS you will get the following error: 

4.7.1 Access temporarily denied. ExchangeDefender requires you to have valid reverse DNS PTR record in order to send mail from [IP]

This is a hard-coded network-wide setting that does not allow for any exceptions. The sender will be notified immediately and will have a chance to setup a PTR record and allow us to accept their mail. For help with how to setup and manage DNS please have the remote sender contact their ISP.

This move will reduce the amount of SPAM you have to review each day and will reduce the number of worms and annoying SPAM messages you have been seeing over the past year or so.

I again want to stress that we did not come to this decision lightly but based on the statistical breakdown of where most of the threats were emerging we had no choice but to defer connections from hosts that do not comply with the basic standards that govern the way mail is sent around the Internet.

As always, thank you for your business and trusting us to keep your mailbox clean.

Sincerely,

Vlad Mazek, CEO, Own Web Now Corp.