I’m starting to see faces and shapes while looking through spam logs. If you stare just right you can see people, shapes and faces made up from the spaces between words. I guess this is still healthy depending on just what or who your mind is seeing!
I realized today that it really bothers me to have unread mail in any of my mailboxes. I have two main e-mail accounts that I use constantly throughout the day, and sync them OTA on my mobile phone. If I look at my phone while out and about, I just have to go through any unread mail, file away things I will keep, reply to what needs a reply, and delete or process any spam messages. I wonder if there is a name for this disorder?
I came across this article in the Ninja newsgroups, and was pleased to see that Sunbelt now has a spam appliance called Ninja Blade. It sounds fantastic and let me tell you, if I had the extra cash to spend $2000 on spam filtering for two people, I’d buy it! While I have not tried this system out, I can tell you that its probably going to be a huge success. I checked out the demo admin interface and its especially good for a brand new product. They will be adding all the “bells and whistles” to it as it matures, but I was very impressed with the features in its initial release.
I have been a long time user of GFI software, relevant to this post is their Mail Essentials for Exchange package. I find it to be a very powerful and easily setup anti-spam system for Exchange. I have had very little trouble with it, and it is packed with useful features. However, recently I had some configuration issues with my spam setup, with rollernet really, not even an issue with Mail Essentials, but it got me thinking about my spam filtration system.
I am now on a quest to find an open source anti-spam solution for Exchange. I’m open to Linux based solutions as a gateway of sorts, but would prefer something that resides on the Exchange server running under Windows. Don’t get me wrong, I have a great respect for SpamAssassin and other gateway type spam fitlers, but it gives the end user a much better experience if the anti-spam software can interact with the user, especially if it integrates with Outlook.
Surely there must be some kind of solution out there I could try. At the very least I might install a few different packages under Linux and route incoming mail through them, and from there go to Exchange for evaluation. I can use server virtualization to allow for an easy evaluation of various types of configurations. ASSP I hear is very good and there was one other package that I found last night that sounds promising. I think it could be beneficial to have an additional layer of spam protection at the gateway level before GFI gets the messages and does its thing. My only concern is false positives. Lots of services and companies on the internet today do NOT have the proper DNS/MX confiugration and even at a more basic level don’t have their network setup right. All these network issues can have a major impact on e-mail deliverability. Its always a risk then when dealing with spam filters that you may block legitimate messages. I am always watching spam logs to ensure that I keep an eye on how the system is doing. If web services and companies would do a little work to get their sytems in compliance with RFS’s for SMTP and DNS, and setup the proper network configuration and mail server options, it would be a much better world for mail delivery without false positives.
Over the weekend I got an e-mail from Dennis Heidner who wrote SPAMLOGS for NoSpamToday. In version 3 of NST, the log parser “spamlogs” quit outputting the subject line of messages in the parsed log output. Dennis has corrected this in an updated version which should be available soon on the byteplant contributions area on their website. I have tested the new version and found that it fixes the problem. Dennis has also added some functionality to check for AUTH Attacks. SPAMLOGS conveniently checks for AUTH attacks and outputs the number of attacks per IP at the end (last column) of the spamlogs csv output.
SPAMLOGS is a must have for parsing the NST spamassassin log file, it turns the jumbled and confusing log file output from NST/SA into a readable and useful .CSV format. Combine his software with the automation utility or scheduled task, and it makes managing the mail logs much easier.
I previously mentioned that I’ve been trying to get the new Sunbelt Exchange Archiver installed for an evaluation and I’ve also mentioned the old “IHateSpam” product and the predecessor “Ninja” in previous blog posts. Here is an update on my status…
Sunbelt Exchange Archiver:
I am still unable to get the archiver to work, my issues at this point are with the database connection. No matter what I try, I can’t get the database connection to function. I finally did get the product to install but now you have to configure everything before it can start the services. As usual the Sunbelt documentation is sub-par and contradicts what support tells you. I will probably have to get a support rep on the phone and do a remote install session just to get the product running.
I upgraded my Exchange servers in my company to the latest build of Ninja which includes their new “STAR” engine. This replaces the old Sunbelt heuristic filter with a definition based system like the cloudmark engine. I was told by Sunbelt that their new engine “does not cause false positives” before I did the upgrade. Pre-upgrade testing showed no problems with system resources such as CPU utilization and spam catch rates were the same as previous tests on the old version. The problem comes in when deploying in production. I found soon after enabling the new engine that we were having problems with lots of false positives and even some internal mail was being filtered and going to user’s quarantine. I ended up having to disable their new engine and things are working much better now. I also resolved an issue with the anti-spoofing feature that was marking lots of external mail as spoofed.
I think in general Sunbelt Software is on the weak side in the following areas:
1. Documentation, frequently I find their documentation is incomplete, does not answer questions users would have upon installing, and contradicts other documentation related to steps in the process and also their support staff directly.
2. Internal testing, I know they test their products before releasing to the general public. However its been my experience that there are always unexpected issues when installing or upgrading any of the three Sunbelt Products I’ve used. Like with Ninja and their STAR engine causing false positives, and marking internal mail as spam when its not supposed to. Not to mention the default configuration causes high CPU utilization on the host server.
Unfortunately there are not many other alternatives to do the job that Sunbelt’s software does. I know there is no perfect software, and with software comes its share of bugs. One last complaint would be in diagnosing errors. I know that in Ninja when we would turn logging to high in order to diagnose problems (and you have to turn logging to high as the system logs only useless information in the low setting), the extra disk activity is a huge drain on system performance. This alone is enough to make users complain. But in order to get any useful information from the software, you have to perform this step. Also, the queue folders often start to build as mail backs up into the queue. Most of the time I am certain this is caused by Ninja or more specifically the SMTP event sink it uses. Mail backs up into the SMTP queue folder and before you know it, you’ve got hundreds of messages stuck and not being delivered. Of course you restart the services and try to clear the queue since its obviously a big deal, but then you don’t get any logging as to what caused the problem. Support has no idea, and tells you to run a snapshot which is useless unless your logging level is set to high.
Ninja also accounts for a large boost in disk activity, and shows a marked increase in the disk queue when viewed in perfmon. This causes general GUI slowness and delays when opening MMC consoles.
I will say that when Ninja works, it works well, but the slightest problem or glitch and your entire mail flow system can be affected. I suppose this is a risk with any spam filter, but we’ve had a long history with Sunbelt products and it seems that the core issues we had with previous version of their spam filter have carried over into Ninja in one form or another.
I just found out that NoSpamToday by Byteplant now has a version of their spam filter for Linux. Take a peek…
I found this page on the spamassassin website today, which is going to be very helpful to me. It lists all the tests SA 3.1.X uses and shows all the default score values along with a link to a Wiki on each test to find out more about what it does.