One of the most important and (as Sod's Law would have it) tedious tasks for a system manager is watching the event logs of his or her servers. Even with a modest number of servers it's a pain in the butt to keep up with the mass of log files on your servers – particularly when you consider that each Windows NT/2000 machine has at least three basic log files (out of the box you have one for each of Applications, System and Security), and possibly more if the developers of the applications you use have defined new event log instances for their particular programs.
Windows users are worse off than Unix users in this respect. We'll come to Unix in a while; let's look at Windows first.
Windows event logs
Amazingly, Windows doesn't provide any useful tools for centralised event logging, so you'll have to find a way to do it yourself – or, more likely, use a third party tool. Because shoving all your log entries to a central location implies potentially vast repositories of data, the preferred approach seems to be to drop the information into a database of some kind – SQL Server being the obvious choice if you're heavily into Windows, particularly because you can use the cut-down MSDE version for free.
There are two potential approaches: either you can make the central workstation pull the event logs from the distributed machines, or you can make the latter push their logs to the central repository. It doesn't really matter which approach you take, though we'd be tempted to take the "pull" approach since it means you have one script for the entire network instead of one per machine.
There are plenty of implementations on the market, both freeware and commercial. Three that we've found are:
Unix event logs
Unix has, and has always had, its own logging protocol, Syslog. From day one, this has been a distributed system, so any Syslog-capable machine can send its log messages over the network to a Syslog server.
The Syslog protocol as defined in RFC3164 (a 2001 update of the original protocol) uses a concept of "facilities" and "severities". Facilities come in 24 different types, and equate roughly to Windows' event log instances in that there's one for system (kernel) messages, several for specific applications (e-mail, FTP and the like) and some for arbitrary local use. As with Windows' event logs, items also have a measure of how concerning a message should be, in this case a value from 0 (slit your wrists) to 7 (debugging information).
Configuring a centralised Syslog service is therefore a case of adding the right lines to the configuration files of each machine such that they send messages to one or more remote servers instead of to local files. It's quick and easy – though it should remembered that because Syslog uses UDP for its message transport, there's no guarantee that messages will get through. If you really need guaranteed delivery, you need to check that your Syslog implementation follows RFC3195 ("Reliable Delivery for syslog") on top of the basic standards.
Combining the two
It's no surprise that products exist to extend Syslog functionality to Windows. Let's face it, the protocol has been around for as long as Unix, and it just works, so why reinvent the wheel? The simplest answer to making yourself an integrated logging repository is to adopt Syslog as your protocol of choice, and to use it to throw event logs to the central server from the various machines on the network. Even if you're a Windows-only house, it's worth bearing in mind that many peripherals (networked printers, firewalls, routers, switches) support Syslog out of the box, and so the chances are that you're already a mixed-log system manager even if you didn't think you were. Of the three products mentioned in the box earlier, MonitorWare has probably the most flexible offering – its EventReporter sends Windows event log data to a Syslog repository, while WinSysLog provides Syslog server capabilities if you want to catch all this logging traffic on a Windows machine instead of a Unix box that has it all already.
To finish, there's one important point that you absolutely must keep in mind: time synchronisation. If you're using stand-alone log files, you can probably live with the system clocks of your various machines being slightly different from each other. As soon as you're collating your log events, however, you need items to appear in the right order in the resulting morass if you're to stand any chance of diagnosing problems that arise. So at the same time as you implement your networked logger, make sure you turn on automated time synchronisation – be it Windows' own or an NTP-based one. And although you could set the clock server from your wristwatch, or from the speaking clock, you probably don't want to, simply because computer system clocks always either gain or lose a bit now and then, and so it'll confuse your logger to death when you reset the correct time by hand. So use an atomic clock – oh, and for the same reason, you'll probably want to log everything based on a fixed timezone such as UTC, instead of letting the server adjust things twice a year to fit daylight savings time.
Find your next job with techworld jobs