Over the weekend I got an e-mail from Dennis Heidner who wrote SPAMLOGS for NoSpamToday. In version 3 of NST, the log parser “spamlogs” quit outputting the subject line of messages in the parsed log output. Dennis has corrected this in an updated version which should be available soon on the byteplant contributions area on their website. I have tested the new version and found that it fixes the problem. Dennis has also added some functionality to check for AUTH Attacks. SPAMLOGS conveniently checks for AUTH attacks and outputs the number of attacks per IP at the end (last column) of the spamlogs csv output.
SPAMLOGS is a must have for parsing the NST spamassassin log file, it turns the jumbled and confusing log file output from NST/SA into a readable and useful .CSV format. Combine his software with the automation utility or scheduled task, and it makes managing the mail logs much easier.
I just found out that NoSpamToday by Byteplant now has a version of their spam filter for Linux. Take a peek…
I’ve been using SpamAssassin for some time now, personally and at work. I’ve been working with SpamAssassin rules to deal with spam that comes through the other filters and makes it to user mailboxes. I’ve been writing these rules in a very static way and they are not very effective as spammers frequently change their spelling. I did a search online and found a few good resources for figuring out how to write better spamassassin rules, but nothing really complete, or written for beginners in plan english. Spamassassin is written in PERL, so it uses PERL RegEx in its configuration. For me, since I am using a version of Spamassassin compiled for use on Windows in the NoSpamToday product, I am editing the local.cf file in my NST installation folder. This is where I add my custom keyword filters.
Before we get started, keep in mind, that I’ll be constantly updating this post with the most recent information.
First, I’ll assume you at least know what Spamassassin is, whether or not you are using NST (NoSpamToday for future referece). In using custom keyword filters, you basically have 3 lines of text for each word, the actual line telling SA (Spamassassin for future reference) where to look (i.e. body, subject, etc). The second line is the description telling basically what the filter is doing. The last line is the score, that SA uses to assign to a message matching the first line of code. I’m going to post the actual conetent of this post as an extended entry, so the main body won’t take up your entire screen. Click the title of this post to read more.
Read the rest of this entry
Today, a revelation has occurred to me regarding custom keyword filters! I use a product called NoSpamToday and was trying to find better ways to write custom SpamAssassin keyword filters (or rules). Normally I can filter for specific words or phrases, but spammers often change their spelling. This could increase or decrease the number of characters in a word depending on what junk filler letters they throw in the mix. I just studied up on PERL RegEx (Regular Expressions), and found that PERL uses a . as a wild card character. This was important, since I’ve tried the & and * with no success. Knowledge of this, enables a whole new level of keyword filtering I can do in NST. For example, yesterday, I received 325 messages with the word pharmacy in them, just spelled 325 different ways. It was always a PHA(three lower case letters randomlyinserted here)RMACY. I can now add a single keyword filter for PHA.*RMACY and filter all of them out with only a single keyword filter. It won’t matter how many letters they put in. And if I find they change the format of that type of slepping, I can simply adapt my rule or add new rules for other forms of spelling. Now I can go through the custom keyword filter and tweak other instances of keywords and make them infinantly more effective!
I made another mail configuraiton change last night on my home mail server. I did have NoSpamToday setup to forward all valid mail to 602LS on normal port 25. I decided to change this to a custom port so that I could setup port forwarding for the new port and use it externally from work or while traveling. This way, I can setup my mail client to use the new custom SMTP port on 602LS and all mail sent to my wife or users on my home network won’t have to get routed out to the internet, through rollernet and then back in to NST and then to their mailboxes.
SRS which can re-write the address headers and work around the fact that the messages were received by a forwarding mail server. I have yet to impliment a solution for this, as my mail servers use three SMTP servers and a workaround for all three is not yet available. Rollernet is working on implimenting SRS functionality on their servers, so I’m holding out for now hoping that when they get SRS working on their end, it will fix my issues of false positives on the SPAM filter. I think this will work in my case as NoSpamToday is my only SMTP server that does an SPF record check. So once its received by NoSpamToday, it should be forwarded to 602LS and accepted as valid mail. I’ll make another post with the results of this configuration once Rollernet releases this functionality.]]>
Microsoft Active Directory:
My home network is built on Microsoft’s Active Directory. I use active directory to organize my user accounts (all two of them), my computer and group policies. With group policies I can set common variables for all my workstations, servers, etc. This way I don’t have to hand configure everything, its all automatic. Group Policies are a great way to manage your network workstations or servers. There are other solutions here, some people like to run Linux at home, and I’ll admit, I do too from time to time. I love linux, but there are still too many apps I use that require Windows. From time to time I demo some of the latest Linux distributions and try things out. I think its great, and if I had a 4th computer to run it on, I’d probably run a linux server or desktop as well. Some people like novell, some people like MAC, its up to you. This is just how I am doing thing. I have group policies set to add customization to my desktop mainly. Things like a browser title, automatic update settings, common software distribution, etc.
Domains, e-mail and more:
I guess I can’t go much further without explaining how I also do my domain names and websites. I’ll write more about this topic later on as a how to and what you should know for getting your own domain and website. But for now, I’ll keep it simple. I own several domain names which I use for various purposes. I have one domain that is for all my server equipment, like my hosting server that hosts my website and some other websites I host for people (for free unfortunately). These servers are in a data center and I simply “rent” the server from them on a month to month basis because its cheap and does what I want it to do. Plus they take care of maintenance and problems. Then I have a primary domain name I used to use for my hosting company’s website. The backend server domain ended with a .net and the primary domain is a .com. These extensions can be anything you like, but I stuck with a traditional format. Then I have a third domain for my personal website which is mainly for my family and my blog, etc. Here is where the bulk of my incoming and ougoing e-mail comes from, the other two domains are mainly for servers and a now closed hosting company. I do have some other domains, but don’t really used them yet. I’ll be expanding that later on as well.
So now you know I have a shared hosting server which hosts my websites and most functions of my domain names. Now when it comes to e-mail, you’d naturally assume this server also handled mail for my domains as well right? If you said yes, you’d be wrong. I’m using a service called Rollernet which is a mail forwarding service. Since my ISP restricts incoming traffic on port 25, it was necessary to setup SMTP on a non-custom port. However, this causes a problem because when someone on the internet sends me an e-mail, most mail servers only send mail on port 25. So if I’m running SMTP on a non-custom port, how do I get my mail? Here is how. Rollernet’s servers are listed as the MX records for my domains. This means, that when you send me an e-mail, its actually received on port 25 by rollernet. They take the mail, queue it, do some scans on it for viruses, spam etc, then they forward that mail to my home mail server on a custom SMTP port. Of course I have this port setup in my cable modem and firewall to allow it to be forwarded to my mail server which resided on my LAN. Now here is the complicated part. My home mail server received mail on a custom SMTP port and is received by NoSpamToday, which is my SMTP level SPAM filter. NoSpamToday (NST for short), filters for SPAM, viruses etc, and basically makes sure that the message is valid before it allows it in to my mailbox. Now NST is not a mail server, its just a SMTP server, so another component is needed here, thats where 602 Lan Suite (LS for short) comes in. NST received a message for me on a custom SMTP port. Once it makes sure that the message is valid, it then forwards that message to 602LS which receives the message on the standard SMTP Port 25. 602LS receives the message and performes a few checks of its own, like scanning it again for viruses, doing aother SPAM check and finally delivering it to my mailbox. 602LS also has a built in webmail server, so I can check my webmail from anywhere in the world. This is also where port forwarding comes in as the ports for webmail need to be setup to route to my home mail server from the outsite. Using my public DNS zone, I can add a record for webmail to my domain, so I can go to http://webamil.mydomain.com/mail and get to my web interface. This way I don’t have to use DynDNS or any of those services, since my public IP on my cable modem rarely changes. Now if it were to change, I’d have to manually update that in my DNS zone. So watch out for that if your using this scenario. I am aware of it and know what to do, so for me its not a big deal, but if your new to this, don’t set this up and wonder why it breaks 9 months later. Keep an eye on your public IP.
Lets now talk about outgoing mail. I don’t know if your like me, but I find myself in situations at work and abroad where I find that my company network or hotel network restricts SMTP servers to their own servers and won’t let you send mail using your own SMTP configuration. For example, at work I run a simple server monitor that sends alerts. But my company has a firewall in place that limits outgoing SMTP traffic on port 25. Now I bet your wondering where the SMTP component from IIS comes in to the picture from my previous post. Here it is. I am running IIS on my mail server but only the SMPT component. So I setup Microsoft’s SMTP service to listen on a custom port (different from my incoming SMTP port for normal e-mail from Rollernet). This way, I can setup my monitoring server to use my custom SMTP server at home to send the alerts. So in my situation, my monitor program detects a problem with a server in my office, it sends an alert to my home mail server on a custom SMTP port. My SMTP server then relays that message to my shared hosting server which then sends it to the desired recipient on a standard SMTP port. This way, I can use SMTP wherever I am, still get my messages or alerts sent and accomplish my tasks. This custom SMTP service is protected by a username and password and relaying with it is denied. Relaying on NST is also forbidden. Ok, so how about my home PC? Ok, simple, we use outlook on our home PC, so outlook is setup to send/receive mail from 602LS through POP3 and standard SMTP. We send a message from outlook, it is received by my home mail server on port 25, which then forwards that mail to my shared hosting server. Some ISPs also restrict outgoing SMTP traffic, so here you may need to setup a custom port on your public SMTP server and configure your mail server to send all outgoing mail over a “SmartHost” or custom SMTP configuration. My shared hosting server then delivers the mail over standard SMTP to the recipient’s mail server.
So in summary, yes this is a complicated setup, and no it may not be for everyone. But I will say this, there is a degree of pride that goes into setting soemthing like this up. Now I’m a Microsoft Engineer, so I’ve been doing networking for a long time. No this is not the way to go about setting up a business or large company. Obviously I’d recommend using Exchange or more powerful mail servers and betters ISP connections. But if your a techie and want to setup a really cool home network, this guide might just help point you in the right direction.
Lets talk remote access. So how do I manage this home network when I’m not home. Easy, RDP. There are lots of people around that don’t like RDP, its not very secure, and has its issues like any other software or technology. For me however, its perfect. I simply forward port 3389 from my cable modem to my firewall and from my firewall to my PC, I can remotely manage any machine on my home network. Now I took it a step further, and actually setup a custom RDP port on my other machines, like my servers and second desktop. This has the advantage of being easy to individually RDP into any machine on my home network without first having to remote into my home pc and then into another machine. In conjunction with DNS for easy naming, its a snap. All you need to remember is the custom port number for each machine. I only have a few so its no big deal, if you have many machines I’d recommend finding a better way, such as VPN. Through RDP I can remote control, and virtually manage any server or desktop on my home network.
Web management: I also use a program called Remotely Anywhere (www.remotelyanywhere.com). Its a great application that runs as a service on Windows. With it, you can remote control, Transfer files, totally manage all aspects of the machine right from a web browser. Its very robust and powerful, with tons of additional features too numberous to mention. Its one of the best web based remote control/management solutions I know of. This can also be setup on a custom port, so it will need port forwarding configured for it as well.
FTP: I used to have a NAS server with FTP setup so I could FTP directly to my RAID5 storage device. Now that its gone, I don’t really use FTP anymore so I removed it. I use an FTP site on my shared hosting server temporarily if I ever need to send anything through FTP. I can grab it from home later.
Internet Access: Because my cable modem and firewall do NAT, its very easy to provide for internet access to my workstations and servers on my home network. The firewall is the gateway on my network, and Microsof’t DNS handles all DNS related operations on my network. My DNS server is configured to forward all requests for external host names to my ISP’s DNS server. It then caches the results and can reply much faster to any requests my workstations or servers make. Internet access is basically a simple NAT solution provided by my firewall and cable modem.
Points of Failure:
With a system like this there are other considerations that need to be taken into account. Amoung them are power, redundancy, damage, replacement, etc. For example, if my power goes out what happens. Well for me I have my critical equipment on a UPS. Since this is a home network and not a critical system, the UPS will keep my servers and internet connection up and running for 5 minutes. This should be sufficient as long as the power isn’t out for long, which is isn’t usually. What if my firewall or cable modem goes bad. Well then I have a problem, as with my ISP I have to have them come and activate a new cable modem. So I’d first have to buy a replacement and then have them install it. This can be done usually by the next day. So what if my mail server or other network equipment is damaged. Well, for mail, if my home mail server becomes unavailable, mail will queue at rollernet, so I won’t loose any e-mail. I can even redirect that mail to my shared hosting server if I wanted to so I could get to it. If some of my network gear fails, it will obviously need to be replaced. I’d try to repalce it with exactly the same modem so that if it had a configuration with it, I could easily restore a backup config file to immediately get my network back up and running.
Security: What about security, how secure is this setup? Very secure. Even considering I have ports forwarded into my LAN from the outside. This often makes security experts very nervous and for good reason, but again, this is not the NSA, I don’t have anything on my home network worth anything to anyone but me. That is not an excuse for having bad security. First, I have a double NAT solution, so even if someone could hack in past my cable modem, they couldn’t get further than my firewall. If they could get past my firewall by some miracle, they would not be able to access anything on my network, since all network traffic between workstations and server is encrypted through Kerberos. The worst they could do if map out my network and find my IP addresses. DOS attacks are also a possability, but there isn’t much that can be done about that anyway. Again, I’m not saying good security isn’t important, and the measures I’ve taken are sufficient for my needs. Please don’t think I’m advocating bad security measures.
Thanks for taking time to read this post, I know it was long. Keep an eye out for more tech posts in the near future. I’ll also post some images giving you a visual of how all this works. Here is a simple visual aid of what I’m talking about above.]]>