A variety of notes on LDAP routing in Sendmail…
MovableType seems to suffer when it gets a comment spam attack.
I don’t know if they’re designed to be DoS attacks or just nasty spam attacks, but when they happen they spike the load average on Heiwa and bring it to its knees. Fortunately today I was already logged in there and had a pretty good idea of what was happening before I could confirm it.
Steps to fix: block access to the web server at the shaper (to get the load back under control), grab the Apache log file and identify the comment spam offending IP address(es) (look for “mt-comment” in the request log), block the offending address(es) at the shaper, and then unblock the web server.
This morning I delved into finally preparing the actual server for SAgate.
I’ve finally made the transition to FreeBSD 5.3, and it seems to be a very nice step forward in the OS. The basics are the same, but it’s got the new tech that we’ve been waiting for.
Mostly I downloaded and built ports this morning, but I was at least able to get Sendmail LDAP routing working as far as I had on Karelia and get a few other ports ready for more of my attention. I seem to need a slight bit of a refresher on LDAP routing, though, as it’s not wanting to accept addresses in LDAP unless I masquerade as earlham.edu, which I’d rather not (for the sake of multiple domains in SAgate, and also for the fact that an MX shouldn’t have to masquerade as a domain that it’s MXing for, should it?).
But dang, that server’s loud. I need to get it out of my office as soon as I can. At least it’s not as loud as the VMware box I had in there for a couple of days, though.
Apparently CS installed milter-greylist version 1.6 last week and has been enjoying it greatly. So I’ll be heading that way with SAgate rather than trying to build a MIMEDefang/perl greylist that does what I want it to.
The main catch is that v1.6 doesn’t support as rich an exception language as I’d like to deal with multiple domains in SAgate. v2.0, which is still in beta, has a much nicer ACL language that can properly deal with positive and negative regular expression match requirements, mix up recipient and IP address ACLs, etc. It looks like v2.0 is stable enough to run with, but I’ll keep my eye on it. This milter also has the nice feature of being able to sync with other peers in an MX cluster — maybe not necessary here, but I’d kind of like to build SAgate with load balancing cluster capabilities, if possible (and I think I can, between MySQL, milter-greylist, and some rsyncing of the quarantine directories).
Well, a little bit of spring break, and then catching up. Monday I spent most of the day researching APC racks and UPSen for the machine room expansion.
It looks like the top of the line NetShelter racks are pretty much standard, and that’s actually the model that we already have, so getting four more of them won’t be a problem. The Smart-UPS RT seems to be the UPS we want to go for — it’s a double conversion online rather than line interactive, which is a step up from any of the other Smart-UPS models. The Symmetra line is very attractive with its redundancy options, but at a higher cost, more rack space, and lower kVA output, I think we can be better served by the Smart-UPS (particularly with a generator and an automatic transfer switch, which comes in relatively inexpensively).
One of the tricky bits is that it’s apparently ill-advised (to the point of not being possible) to connect a UPS unit with greater than 5kVA to the power source with a plug — you have to hardwire it. For general ease, I therefore limited the discussion to the 5kVA Smart-UPS units, which at one per rack would give us a total of 20kVA power in the rack system.
Spent some time this week with a spare V120 and Sun Java System Messaging Server.
I swear, the iPlanet/ONE/Java System family of messaging server products are based on Cyrus, but I can’t find anything that actually comes right out and says so. It’s just that the message store directory structure is too close to be a coincidence. That, and the fact that other Sun enterprise systems are based on open source software — like Directory Server being derived from the UMich LDAP server.
I installed Messaging Server 6 on SITH after decommissioning it from use as the old WebDB server. Since this is a test box and the setup of the whole system is somewhat interdependent on specific versions of Sun enterprise software, I didn’t try to get it talking with our LDAP directory and instead used the bundled one on SITH. The install, as far as that goes, is pretty straightforward. The main drawback is, as I say, its interdependencies on other software and its approach of assuming your a totally standard Sun enterprise-only shop. This will make it play less well with others, in particular Mailman, SquirrelMail, and possibly SAgate.
As you might expect from the underlying structure, the raw performance of Messaging Server is pretty zippy. I wasn’t able to perform rigorous tests, but I see no significant difference between it and Cyrus.
In any case, it sounds like the latest word is to strongly consider Cyrus. I’ll be doing some stress tests on Cyrus after break in order to determine what kind of hardware we’ll want to use on it.
After clearing up some of my inbox, I spent most of the day writing some utilities for dealing with Mailman.
One script that synchronizes Mailman aliases with LDAP, both for future LDAP routing and for current use in getting these aliases over to KE in a simple fashion. And the other script was for extracting aliases from LDAP so that non-LDAP-enabled Sendmail can get them in alias and virtual user files.
After a little longer than I was anticipating to rebuild list archives, I’ve got Mailman upgraded.
There are a few archives that I still need to rebuild, but they’re not critical and I’ve got them rebuilding in the background. There may be an issue with people sending mail from USER@ke.earlham.edu rather than just earlham.edu, but people shoiuldn’t be doing that anyway. Although I may have to change some masquerade settings on KE to make that work. Everything else seems to have gone as planned.
I got the rewrite of the MIMEDefang filter for SAgate mostly complete today and have been testing it a bit. It’s looking good.
Naturally, a bunch of errors the first few times, but thanks to MIMEDefang’s -test mode, I got the bulk of those straightened out. There were a few I had to pin down with syslog debugs to figure out what was going on.
The main things left at this point are putting in more logging of the appropriate types (primarily in filter_end() for SpamAssassin status), making the PHP web interface understand the XML config file, and testing, testing, testing. Some command line config tools wouldn’t hurt, either.
I’d like to get a greylist implementation fitted in, but that may come in round two. I have, at least, found some good greylist reference implementations that use both BDB and MySQL as backends, so attaching that wouldn’t be too hard — just have to fit it in.
Cleaned out a few trackback spams.
Apparently I’d left pings turned on for a few posts. Silly me. I usually mean to disable comments and trackbacks in here, since I’m really just talking to myself.
So this post is partly to see if deleting those trackbacks will make ecto happy. It can’t seem to update its list of recent posts, nor figure out that it actually posted the one yesterday.
Revisiting SAgate and rewriting large parts of it.
I can never seem to get MIMEDefang filters exactly right, less so with ones that I’ve never put into production.
I’ve written the SAgate config file in XML, parseable with XML::Simple. SAgate will operate in MX-only mode, meaning that we won’t tell our users that they should use it as the SMTP host (or POP or IMAP host with it doing proxy, either). This means we can simplify a lot of the logic to figure out whether we should apply filters (particularly SpamAssassin) to any particular message (it had been based on a munging of recipient).
I’ve introduced the idea of multiple domains for SAgate to operate under, and built a robust routine to identify which domain is active for a particular message/recipient. I’m also building in logic to have the whole filter for a domain turned on or off (or logging-only) with exceptions for particular addresses.
All of this should end up making it more robust, slightly more flexible, and actually more simple.
Started playing with Sendmail queue groups this morning, and I’ve got a queue group set up on KE for the local mailer.
The following needs to be added to the mc file:
QUEUE_GROUP(`qlocal', `P=/var/spool/mqueue/qlocal, F=f, I=1m')The directory /var/spool/mqueue/qlocal needs to be created (and it must be below /var/spool/mqueue). Restart sendmail, and it’s happy. You can see the contents of that queue by specifying -qGqlocal for mailq. Run that queue the same way with sendmail to process it by hand.
The high power way to do this would be to play with the filesystems that the queue directories are on.
Expanding on yesterday’s work with the proxy cache in OpenLDAP, I have a fully functional system for caching the Sun ONE entries for Sendmail LDAP routing.
This config file gives the functional basics for a working proxy cache system for this setup:
And this file has the necessary schema to get the object classes and attributes from Sun ONE (iPlanet) into OpenLDAP:
The schema is very bare bones and does not implement all of the Netscape mail attributes and object classes. That’s ok, because I’m overloading the Netscape stuff for our own purposes in Sendmail routing rather than for Netscape/iPlanet/Sun ONE Messaging Server. See the entries from last September (aliases and routing) for details on how I’m doing this.
The purpose of this proxy cache is so that the MX that’s a Sendmail LDAP routing client doesn’t have to hit the master LDAP server constantly to figure out how to deal with known addresses. Instead, it should hit it the first time the address appears, and then keep it in local cache for a while (in the config file above, 30 minutes) before having to ask the master again.
Going back to the old OpenLDAP proxy cache stuff, this time for the SAgate spam gateway, so that LDAP mail routing doesn’t hit ASHTI too hard.
The trick is that OpenLDAP doesn’t have a definition for the mailRecipient object class, and it’s tricky to find one out there. Details to be inserted later, but the basic idea is to just get one in there that requires cn and allows mail, mailHost, and mailRoutingAddress. Once these are in, OpenLDAP will happily procy-cache the query that Sendmail makes back to Sun ONE.