As Indiana is observing Daylight Savings Time for the first time in 30+ years this weekend, we have had to update all the time zone settings for the servers. All servers are observing the new rules for Indiana time that have DST starting on April 2, 2006.

• Solaris patches are available that incorporate the new Indiana time zone rules: 113225-03 for Solaris 9 and 109809-03 for Solaris 8.

• Purdue has a good overview of what to do about the new time zone information. In particular, they provide instructions for compiling new zoneinfo files for Unix and a link to the source files to do so. I have done this for all the FreeBSD servers.

It's time to get this log back on track. What follows is some updates on a number of the servers. See also the TWiki reference for each server on its archive page. TWiki should contain the latest general description of the server, while this log will contain more historical markers of significant changes to the server.

Today we are rebuilding the PacketShaper for the student network.

Late last fall we rebuilt the main campus PacketShaper. This proved beneficial in conjunction with the upgrade to the device's firmware and allowing it to clear out old information. This week we are rebuilding the student network PacketShaper with the aim of improving shaping performance and management on the student network. The PacketShaper will be in learning mode until the end of the week, identifying the primary application protocols being used on the student Internet connection. During and after this wee will be assigining service levels to these protocols to best allocate the bandwidth.

The PacketShaper can’t seem to recognize passive FTP any more, so I removed the complicated FTP classifications.

We had 6 FTP classes, corresponding to inbound and outbound clients, servers, and general. The general was not allowed, and the servers were limited to those on a host list and some subnets in CS. However, passive FTP was getting classified under the general class, since it looked like active FTP to an internal server not in the host list. Until the PacketShaper recognizes passive FTP again, we’re simply allowing all FTP and giving it the policy that the client and server FTP classes had.

Yesterday I rebuilt the main campus PacketShaper to solve some lingering problems we've been seeing with a number of protocols.

I think this came from upgrading the firmware earlier this fall. We were seeing problems with FTP transfers in particular, although I had noticed problems with SSL versions of IMAP and SMTP, and others had been having (possibly unrelated) problems with the library proxy system. Resetting the shaper made it crash twice on every boot with an invalid configuration error. Since the error was unhelpful in the extreme, I opted to completely rebuild the shaper from factory defaults.

Everything seems to be working properly, again. A side benefit of this is that we can clean out the cruft that has accumulated over the past couple of years.

I installed a libxml2 package on PACO and ROJ for development purposes.

Used the GPlxml and GPlxmlx packages, version 2.4.25 for Solaris 8 from Gary Pennington.

OpenSSH has been upgraded on all the FreeBSD servers to the latest openssh-portable package, correcting the vulnerabilities discovered last week.

FreeBSD 4.4 packages were installed on HEIWA, KE, and SHANTI. A FreeBSD 4.7 package was installed on PAX. These are all openssh-portable-3.6.1p2_3. They install into /usr/local and require the following changes to /etc/rc.conf:

sshd_program="/usr/local/sbin/sshd"

sshd_flags="-f /etc/ssh/sshd_config"

PAX is the only server that currently required these changes, as the others had previously been upgraded to OpenSSH-portable. PAX also required minor changes in the /etc/ssh/sshd_config file.

While we had few, if any, incidents of the now infamous Blaster worm, we just experienced the effects of the ill-conceived Nachi worm. This worm supposedly cleans up and patches systems that are vulnerable to the same exploit. This is a striking example of how vigilante justice and the desire to write "cleanup" worms is, if anything, worse than the original worms.

The Nachi worm exploits the same vulnerability that the Blaster worm uses, however its aims are to cure the affected computer. It attempts to download system patches from Microsoft, apply them, and then spread to other vulnerable computers, fixing them. Unfortunately, its method of spreading uses ICMP "ping" packets to map out the network. This causes severe congestion on local area networks and renders completely unusable wide area networks such as the college's T1 line. Thanks to an emergency firewall device, we were able to isolate the infected network and block these ping packets, using both the firewall and the campus core routers. We are currently in the process of cleaning up the systems infected with the "cleanup" worm.

[Note: "Lovsan" is an alternate name for the Blaster worm.]

We've seen a lot of Windows RPC traffic recently, probably as a result of exploit code published on the net to attack the vulnerability.

This graph shows the inbound traffic for Windows RPC protocols as recorded at our network border (the main campus PacketShaper) for the last week:

Scanning has increased radically over the last day. All of these connections were dropped or ignored; as of this morning, the PacketShapers are configured to block all inbound and outbound Windows RPC protocols.

The latest Vexira virus definitions should detect viruses that search for this vulnerability.

We upgraded the main campus PacketShaper to a 10 Mbps link size capability and upgraded the dorm network shaper to version 6.0.1 of the PacketWise software.

The 10 Mbps upgrade will allow us to use our planned bonded T1 lines (two total, for a 3 Mbps link size). Previously we were limited to 2 Mbps - sufficient for a single T1.

The new PacketWise software includes more sophisticated traffic shaping and a number of management enhancements. We will be installing it on the main campus shaper early next week at a low usage time (it will take only a couple of minutes to upgrade the software).

Update, 8/12/03:

The main campus PacketShaper is now running the latest PacketWise software as well. In addition, I adjusted the rules to completely block all Windows file sharing protocols in response to the W32/Blaster worm and MS RPC calls.

I replaced the hubs forming the private server network with a Summit 24e switch.

The Summit is reclaimed from one of the dorms after replacing the dorm hub and switch hardware with Dell switches. The private net is now fully 100Mbps switched using good hardware.

Oh, right. The UPS has a new emergency bypass switch.

Not much to say here. Lee Higgs from Melling installed it, and Mike Kammer from Liebert verified it. The UPS twitched slightly a couple of times when large loads were put on it (RAID arrays or the Black Diamond), but is doing fine now.

Be sure to turn the back switch from UPS to BYPASS before turning the bypass switch on the wall. Otherwise, everything's quite nice.

The firmware on the Cyclades TS2000 terminal server was upgraded from 1.3.4 to 1.3.6 today.

Some problems arose with the RADIUS authentication: the files /etc/raddb/server, /etc/pam.conf, and /etc/motd were not in the config_files list and were not saved. Changing these back to appropriate versions worked.

Point to remember: it is a good idea to tar up the /etc directory and save it on another system before upgrading the firmware. This gives a good backup of all the configuration files in case the upgrade procedure touches more than the release notes say it will.

Rather than using the FTP download for the upgrade images, I downloaded them to my workstation (via FTP) and then copied them to the TS2000 using scp from the console.

The Sola 5000 UPS blew some circuitry this morning, prompting a repair call and moving the upgrade up.

This looks similar to the problem it had shortly after installation when some circuitry blew out. The repair technician has ordered replacement parts for express delivery to Dayton tomorrow morning. While he's here doing the repair, he'll also install the 12 KVA upgrade module. We'll still need to schedule downtime to install the maintenance bypass switch as planned.

I've set up a private network for the servers using secondary ethernet cards and a 10.18.0.0/24 subnet.

I'm currently using a 5 port 10/100 hub in the APC rack. All the servers have at least one spare 100 Mbps port. I'm using static /etc/hosts files. In the hosts file, all the servers that also have public IPs are spelled backwards (e.g.: PAX is XAP/10.18.0.2). The primary reason for this is to put the new CD server on the private net. It will also allow us to push other data (possibly backups or other private info) across this net without having to rely on the core switches and possible compromise there.

Upgraded Samba on all systems except MIR on Friday morning.

PACO and ROJ are using Sunfreeware.com packages (requires the popt package). All others are using FreeBSD packages built on my workstation.

On installation on SHANTI, it somehow overwrote all individual entries in the smbpasswd file such that passwords were null and accounts were disabled. Restored from previous night's backup.