How our SPAM detection systems work
ExchangeDefender SPAM protection is a layered process that includes many components which together determine if a message is SPAM or not. While there are too many layers to list and the combination of what we use changes on an hourly basis, we wanted to share some detail of how our process works so you can understand it and explain it if necessary.
RBL – We use several free, commercial and our proprietary realtime blacklists that contain IP addresses of known spammers. These IP addresses typically belong to dialup, cable and DSL/fiber client addresses that shouldn’t be running a mail server, compromised/infected mail servers, unused IP ranges, compromised workstations and devices. These IP addresses account for a bulk of all SPAM mail and most of them will never even make it inside of the ExchangeDefender network.
Reputation blacklist – We use commercial and our proprietary reputation blacklists that indicate how trustworthy the IP address is – have they sent us a lot of SPAM in the past? Are they suddenly sending us hundreds of thousands of messages if they only accounted for 10 in the past month? Do they have legitimate DNS, reverse DNS.
URL blacklist – We use several commercial and our proprietary URL blacklists that identify web site addresses used in previously confirmed SPAM. If you’ve ever received a SPAM message you know that it had an external image or a link to a web site: we look at all web site links in an email and compare it to a list of known SPAM site targets.
Distributed checksums – We use several commercial and proprietary statistics models (warehouses) to determine the likelyhood of bulk mail. Because the only way spammers can distribute mail efficiently is through massive blasts, nearly all message bodies are identical. Each message has a unique signature (an MD5 checksum) that can be compared with other messages and when identical checksums are found it’s more likely that bulk mail is being sent.
Proprietary header checks – We use proprietary header and message checks to determine if the message is a part of an existing conversation between a third party and our own client. We also check if messages have been spoofed or if they have made their way through several gateways, if the language in the message does not match the language of the machine it was sent from, etc.
SPAM keyword and heuristic checks – We use a wide array of SPAM characteristic checks that take into the account the size of the message, subjects, fonts and images. For example, legitimate email typically doesn’t come without a subject or a subject that contains a lot of special characters, it doesn’t come without a persons name in the From line or other “weird stuff”. The “weird stuff” category is so wide and so contextual that it takes most of the time to process.
Now that you are familiar with some of our processes you’re probably getting the idea that SPAM filtering is very similar to the way virus scanning works – we use patterns from known SPAM messages and existing spammers to build a statistical model that tells us if a message is SPAM or not. It also explains why certain messages are delivered instantly while others may take up to a minute or longer to process (delayed header checks, suspicious web site in the email, conversation thread checks, temporary DNS failures or large attachments that require scanning)
We build most of our proprietary infrastructure based on your feedback – we look at the pattern of messages that you release which builds a model that adjusts scores along the way specifically for you – if you release a TON of messages with impotency drugs you are more likely to receive a Viagra message than another user that only releases financial newsletters. This is why your feedback through ExchangeDefender Outlook Addin is so valuable to us. When you hit “Report SPAM” that message is dispatched to us and reviewed by a live human being that generates a scan rule to eliminate that specific message in the future.
ExchangeDefender SPAM engine uses third party scanning engines with realtime data feeds and our own proprietary engine is updated hourly.
That may seem excessive but keep in mind that spammers adapt their message content for each batch – adding a different subject, different web site, different spacing and subjects every few thousand messages. In order for us to keep up the need for both the realtime update and constant reengineering of the SPAM engine itself is cruical to eliminating as much of the annoyance as possible.
How come SPAM messages slip through?
There is no such thing as a “slip through” when it comes to SPAM, all mail that is not sent by a sender on an RBL that passes our virus scanning is considered legitimate until proven otherwise. Tens of thousands of checks later (within a split second) calculate a score that identifies if message is SPAM (90% confidence) or SureSPAM (99.9%). There is no person on the gateway reading each message, the score is assigned by the computer based on the statistical model – so even if you looked at the message and could clearly tell it’s SPAM, the artificial intelligence is not quite there yet.
How come legitimate messages end up in SPAM quarantines?
Bad scanning methodology. For example, say you’re receiving a lot of “lotto” SPAM. Someone at ExchangeDefender may adjust your model to get really aggressive when it encounters the word “lotto” in the subject or headers. It works flawlessly, until you are sent a message by your business partner and the message is also copied to Lisa Lotto. This is clearly an oversimplification but it happens.
How come updates happen every hour and why does it take so long sometimes to make changes?
Most of the implied delay is due to the size and the scope of the ExchangeDefender network – to protect millions of people we have a very large network and very sophisticated layered infrastructure. Smaller updates are done very frequently (within an hour) but are staggered (happening at the different minute of the hour) because during engine reloads individual nodes cannot process inbound mail. If we restarted the engine on every node at the exact same time we’d basically shut the network down for a few seconds every hour.
Some large-scale changes (when we add a plugin or change our model analysis or whitelists or blacklists or new detection procedures) can take on the order of weeks or months. Typically we don’t deploy new software across the whole network – it is done in stages to eliminate anomalies in the deployment. Working with large scale distributed systems is different than managing individual servers, clusters or even networks: Changes to a globally distributed network with linked load balancers (while great for redundancy and service availability) is a challenge that requires careful monitoring and rollout procedures to minimize destabilization across the whole network.
We get hack attempts and DDoS attacks thousands of times an hour – the systems have an automated process for dealing with that activity – so when we roll out new software that makes the network overload quickly (and make it think it’s under attack) we have a big mess on our hands. This is why the rollouts are staggered and have a procedure that is at times excruciatingly long (especially if you’re sitting in the NOC watching the queues and your blood pressure goes up with every uptick in the percent utilization)
How come whitelists don’t always work and I have to whitelist the whole domain?
This is an inconsistent behavior that is reported from time to time and it has to do with the senders email address and client software. ExchangeDefender looks at the From line (the same one you see in Outlook) and whitelists only the senders actual address. This works 99.9999% of the time.
Some senders use impersonalization or sending on behalf of a different email address or use specific taglines in the From address that end up being randomly generated for each email. While Outlook may see message from Vlad Mazek <vlad@ownwebnow.com> the message itself may have come From: vlad+2381ekr259@ownwebnow.com with a fingerprint used to track read receipts.
Long story short, if you have a person that is using these systems or CRM to alter the message for tracking/sales/marketing purposes, you can’t trust their From line for a whitelist, you will have to whitelist their whole domain. Good news is, this is typical for smaller domains and you’ll never be whitelisting all of @aol.com or @gmail.com
How come there are delivery delays?
Our support team can help you with that.
Half the time the issue is related to the DNS and the other half is due to the temporary network or Exchange issues on the client side. Almost all of the tests we have done for our clients fall into these two categories and almost all of the things you can do to minimize delays are outlined in our deployment guide.
In the event that the issue is our fault due to network congestion, filter failure, virus scanner malfunction, DDoS, routing issue, misconfiguration, etc – the issue is noted at http://www.exchangedefender.com/noc – please subscribe to the feed or @xdnoc or post your cell phone in our portal – we text, tweet, blog and (everything except email, for obvious reasons) notify you of any ongoing network issue that might impact service.
So much for tech..
Technology is only a piece of the whole puzzle.
This is going to sound cliché but I absolutely mean it 100%: At ExchangeDefender killing SPAM is our passion. Every piece of junk that passes through our network is unwelcome and we have people here around the clock working on eliminating it from making it’s way to your inbox. It’s a human process and it’s a technology process and the very practical implementation of artificial intelligence that learns and adapts in realtime to fight with SPAM. But just like training a dog takes time, training a computer is a challenge that requires consistency and strict implementation of the process, rules, management and monitoring. With millions and millions of messages passing through our gateways, even a slight insignificant modification can impact SPAM statistics models. It’s not something that’s “broken” that could be “fixed”, it’s just a process of continuous training that we take very seriously and enjoy very much.
Thank you for your business.
Sincerly,
Vlad Mazek, MCSE
CEO, Own Web Now Corp
vlad@ownwebnow.com
Workflows
The majority of my job is talking with partners and I do that everyday. Some, I talk with multiple times a week. Suffice to say, I have come to know a lot about your businesses and how they operate. With that said, this may sound familiar to you.
You are not operating your business correctly.
By that, I mean you have no workflow established. Many of you provide your clients with multiple ways of contacting you directly anytime they wish—this includes calling your cell phone and home phone! While I can go into a list of reasons why that is wrong on so many levels, let us just stay on point for a moment.
Let’s talk about your Support Department for a moment. Say as an example, Client A calls you with a email problem in the middle of you working on Client B’s issue. You take the call, confirm that they cannot send or receive email, and tell them you will handle it as soon as possible. After Client B’s issue takes longer than expected, Client C sends you an email stating they need to set up an appointment for you to address BSoD error on their Accountant’s workstation. After reading Client C’s email, you immediately receive a call from Client B stating that the malware you thought you just removed from their workstation has returned and they can not access network resources, including the Internet. I am writing this and I am already confused!
All the while, you are subscribed to the “simplest and most effective IT Management Platform and customer relationship management system—unifying support, services, management, reporting, accounting, and sales activity under a completely free branded experience.” Shameless plug, I know. 
What is my point? Create a proper workflow for your support department. In fact, create workflows for all your departments within your business. I don’t care if you are a one-person shop either. If you have not clued in yet, your company (however small or big) still has the same major keystone that every business is comprised of: Marketing, Sales, Service/Support, Accounting, and Human Resources.
Creating workflows allows your departments to run efficiently. More importantly, share these workflows with your clients. After all, you have no right to complain if you have not established a set of rules for them to follow.
Michael D. Alligood
Partner Sales & Support, ExchangeDefender & Shockey Monkey
michael@ownwebnow.com
(877) 546-0316 x707
Happy Valentine’s Day!
Happy Valentine’s Day to everyone! Don’t forget to take advantage of our Valentine’s Day promotion this month!
Please take advantage of this promotion because Hosted Exchange pricing will be going up when we make the switch to Exchange 2013. The promotion will be valid on any new account signed up within the promotional period. The promotional period for this promotion will be February 1st – February 28th. So don’t hesitate! Simply use the coupon code that received in the mail or write me an email at stephanie@ownwebnow.com to receive the code. If you have any questions just let us know!
Stephanie Hasenour
VP Marketing, ExchangeDefender
stephanie@ownwebnow.com
Carving out a Niche
Sometimes IT companies can get caught up in all of the technology on the market. This latest solution, that latest service, this up and coming trend, and that gotta have device. Though diversifying your solution stack and providing many solutions appears to be the wave of the future, it is still in the best interest of IT providers to carve out a specific niche.
A niche or expertise in a specific solution, product, or vertical can help to provide an edge over competition and create demand for your company. True IT generalists may have a harder time finding prospects and clients that just need general work. This is where a niche comes into play.
Finding a niche can be as easy as doing what comes naturally. If you are naturally good at a specific piece of software, device, or service it is easy to transition to an expert status and exploit that expertise.
If there is a vertical that you have gained trust in, utilize that niche to drive more customers to your business. The key is in marketing your specific niche. Find the need, and market to that segment. As you gain customers looking for your specific expertise, ask them where they would advertise or who they would get involved with. It is easiest to find someone in the know and get information from them to formulate a good marketing strategy.
As you build your client base, make sure to get testimonials about the specific expertise you are building. This will add to the trust factor and help you to gain a reputation for your particular niche.
Building a niche can be an easy endeavor if you have a place to start. If you don’t have a specific function that stands out, look at your previous clients and find something that has interested you in the past. Find something that you can become passionate about and build your expertise around that.
Frank Gurnee
VP, Channel Services, ExchangeDefender
(877) 546-0316 x4777
frank@ownwebnow.com
ExchangeDefender Billing Change
As many of our partners may have noticed, there have been some recent changes to the normal ExchangeDefender billing system. This month was the first month we ran billing through the new system and some things were a little different. For one, you will now receive a receipt confirmation of the charge that has been processed. We did experience some issues with invoices going out, but we have resolved that and everyone will receive their invoice for March as usual on the first. So moving forward, beginning in March, you will receive two emails, one being the invoice and the other, the receipt of a successful charge.
The invoices may always be accessed from the support portal and now you can choose to view the invoice, or print the invoice. We have also changed the way that ExchangeDefender accounts are listed on the invoices. Previously, the domain would be listed and the price per account. We have now changed it so that it lists the domain name, the number of accounts, and the price per user. Lastly, when you click on the Invoice tab in the portal, you will see a status beside the invoice. The status will either be, paid, past due, or processing. If the invoice is listed as processing, please give us a call because there may be an issue with your account.
If you have any questions, please feel free to contact the Billing Department directly at: 877-546-0316 x 739.
-ExchangeDefender Billing Department
Why Industry Awards Are Important for IT Solution Providers
There are many industry awards throughout the year for IT Solution Providers to get involved with. Some IT Solution Providers may look at awards as pretentious or self-promoting, but the reality is awards can help to drive a business to the next level.
There are several ways awards help business growth.
Recognition
Awards can help bring a spotlight on an IT company. With recognition comes easier access to vendors, media outlets, and distributors. Recognition can also lead to event participation, speaking engagements, and even lead to being recognized as an industry expert.
Validation
Industry awards can help validate or give credibility to IT companies. By listing industry awards on materials and the company website, potential customers are able to easily see involvement and participation, providing them with more information to make an easier decision.
Promotion
Awards provide an easy outlet for promoting the company. It is easy to get local or national attention through industry awards. Press releases are a good way to promote through the web and many local newspapers and TV stations will jump at the chance to run a local interest story.
Receiving industry awards can easily boost company moral and can spark new interest from customers.
Award season is currently underway and there are many awards for IT companies to seek out. One to check out is the Looks Cloudy Channel Choice Awards. Nominations are open until February 28th, 2013 and the award provides a great opportunity for IT industry personnel to get involved.
Most industry awards have an easy nomination process and taking the time to get involved can really pay off in the end…
Frank Gurnee
VP, Channel Services, ExchangeDefender
(877) 546-0316 x4777
frank@ownwebnow.com
Face-to-Face Communication
I am going to forgo my normal blog post today and talk about an article I read recently. Over the weekend while cleaning up my office, I was thumbing thru last year’s magazines and came across an article titled, “The Importance of Face-to-Face Communication at Work” by Chuck Martin. Mr. Martin cited a survey that shows “67% of senior executives and managers say their organization would be more productive if their superiors communicated more often by personal discussion.” With the advent of technology in the work place, email has proven to be the leader in communication.
However, further reading uncovered that many respondents to the survey stated that, “Personal discussion is the foundation of communication.” While Mr. Martin states that the majority of business leaders say their organization would be more productive with more personal discussion, most slip back into the easy, and sometimes impersonal, technology communication such as email, instant messaging, and texting. The article did then go on to state that technology communication is not all evil – it just has a place. It explained that if personal discussion is not warranted, then electronic forms of communication can be used as a choice. The article closes with stating that in the long-term, face-to-face communication assures a clear and concise message coming across.
While sitting at my desk, reading this article, I personally think this article deals straight to the point of what I believe is lacking in small to medium businesses today: Proper Communication. I also think that it is up to upper management to establish proper forms of communication within their organization and to make sure that their employees know these acceptable forms. Every organization should have an outlet for both managers and employees to engage in open and honest face-to-face communication. And while the article primarily discusses the communication relationship between management and the employees; it is important to expand this further into the role of partner and client.
Technological communication such as email, IMs, and texting all require that the technology is working properly when you press the send button. However, this is not always the case. And many people shirk situations when they have to talk to another person directly. This is because they have lost their basic communication skills. I am not saying we should frown upon using technology as a form of communication. What I am stating, or rather agreeing with, is the premise of this article: Know when to use technological communication and know when to use face-to-face communication to mutually benefit both parties.
Michael D. Alligood
Partner Sales & Support, ExchangeDefender & Shockey Monkey
michael@ownwebnow.com
(877) 546-0316 x707
Balancing the Controlled Burn vs. the Forest Fire
This week I have had bouts with managing my time between scheduled items that are improvements to our infrastructure and the “Hey, this is the single most important thing in the world to me” items. I know it’s the nature of the beast. So, I guess I have to figure out how to balance the politely ignore unimportant request versus putting a DND sign on my door.
So this balancing act gets extra tedious in non-ownership management roles. Because in this case, you have items falling on you from above, sideways, upwards, and diagonally. They will come in all levels of importance and from seventy different directions. They will come far and wide, from your boss, from employees, from clients, from vendors, from fellow managers. All while in the middle of a fire. How do you say? “Not now”. That part is easy, but how to say it without sounding like a complete subhuman species of ill manners.
Here’s my concern, folks will appear to be ok with pushback but most deep down feel like they are getting blown off merely because you don’t want to do what they ask. When in reality, however big you perceive your problem to be, the one I am working on instead, is just ever so slightly bigger. And not only that, I find myself in this pattern: If I stop what I am doing, then my controlled burn can quickly turn into a forest fire that I then have to put out myself!
How do you handle this?
Carlos Lascano
VP Support Services, ExchangeDefender
carlos@ownwebnow.com
(877) 546-0316 x737
It’s That Time Again!
It’s that time again…Promo Time!!! Check your mailboxes! We are running a promotion for Valentine’s Day that will be active throughout the entire month of February!
We love our partners so we are going to “Spread the Love” and offer our wonderful partners yet another great discount this month!
We are quickly approaching the switch to Exchange 2013 so this promotion will reflect the LAST chance to get our Hosted Exchange platform at this great price. Each account that is created within this promotional period will be grandfathered in at that price for the duration of the account.
The promotion will be valid from February 1st – February 28th! Get the coupon code from the promo card that you will be receiving or contact me directly and I will give you the code! Always stay tuned to the ExchangeDefender Promotions page for the latest promotional offerings.
DON’T MISS THIS PROMOTION!!!
Stephanie Hasenour
VP Marketing, ExchangeDefender
stephanie@ownwebnow.com
The Disillusion of Business
Let’s start out with a very simple question: By observing the image below, which line is longer?
Most of you recognize this as the Müller-Lyer illusion and know through education that both lines are the same length. However, you would be wrong. If you notice in the image below, you will see that I changed the length of one of the lines.
I would venture to guess that the majority of you didn’t notice the change and proceeded to answer the question they way you were indoctrinated. You did not stop to think that a change could, or would, or should be made. You saw what you thought was the Müller-Lyer illustration and answered accordingly. However, if you would have stopped and preformed a quick calculation, you would have noticed the change in line length. This leads me to my second, and more important, question: Are you making the same uncalculated decisions in your business?
Business, technology, and education are all constantly changing. It is important for you to realize these changes and adapt to them in order to maintain relevance in today’s market. By going on what you have always believed to be correct, you will inevitably be incorrect. Although from a fictional movie, a line from Men In Black can easily sum this up: “1500 years ago, everybody knew that the Earth was the center of the universe. 500 years ago, everybody knew that the Earth was flat. And 15 minutes ago, you knew that people were alone on this planet. Imagine what you’ll know tomorrow.” The business world will change the rules on you every time—and at any time. Learn to recognize these changes and adjust your business accordingly. Retrain your mind into performing quick calculations on items it already believes to be evident. Because if you don’t, you could go from reading a blog about measuring lines to standing in the unemployment line.
Michael D. Alligood
Partner Sales & Support, ExchangeDefender & Shockey Monkey
michael@ownwebnow.com
(877) 546-0316 x707