One of the key functions within any distributed processing system is the passing of code and data between the various devices taking part in the processing task. Although at the fundamental level it's not difficult to make (say) a TCP/IP connection from machine A to machine B in order to send some information, the concept rapidly becomes hideously complex when you start to worry about servers failing, connections dying in mid-transmission, differing data representations on the various nodes (which is likely in a heterogeneous network) and such like.

Commercial message queueing software, in the context we're interested in, does not mean e-mail transport. We're talking about software that exists to remove the pain of application-to-application communication, by "abstracting" the problem of getting stuff from A to B so it becomes a black box as far as the programmer is concerned. So instead of the developer worrying about how to reliably get a chunk of data from one machine to another, he instead buys a "black box" into which he can pour the data, safe in the knowledge that it'll safely reach a suitable destination. The concept is no different from relying a reusable code library in traditional developer parlance, except that the range of stuff that can go wrong in a distributed system is much greater, and so the message passing system is a more complex beast.

There are three major commercial message queueing systems that tend to come up in polite conversation. First is Microsoft Message Queueing, known colloquially as Microsoft MSMQ, which ships built-in with Windows 2000 Server and Windows Server 2003. As you'd expect with Microsoft, you get all the necessary developer APIs with the company's programmer tools such as VB.Net and C#, and it runs on a Microsoft-only platform.

Next on the list is IBM WebSphere MQ . WebSphere MQ was formerly known as MQ-Series, but as IBM has collected more and more of its functionality together in the WebSphere architecture, so the message passing software has been brought into the mix to provide core connectivity between software components. This is probably the best-known mainstream product, and rather than being just a part of an operating system it's a package in its own right. Platform support is also much more extensive than with Microsoft's offering, and in addition to Windows the options are AIX, HP-UX, IBM i-Series (AS/400 if you're old-fashioned), Linux (on both Intel and z-Series hardware) and Solaris.

The third popular messaging system is Rendezvous from TIBCO. Like IBM's offering, Rendezvous is designed as the underlying messaging system for a suite of integrated business automation tools that run on a variety of platforms, and so there's a similarly extensive range of platform support: AIX, FreeBSD, HP-UX, Linux (32- and 64-bit Intel hardware and System/390), OS/390, OS/400, Solaris, Tru64 Unix, VMS and a number of Windows flavours).

In addition to providing the basic function of shipping messages from A to B, message queueing systems are designed to deal with all the possible problems with intercommunication between distributed systems that we mentioned earlier. The core of the message passing layer can be configured such that messages are re-routed to deal with link failures, or even passed to secondary destinations in the event that the intended recipient station has failed for some reason. In such cases it's the queueing system that handles retries, and which re-orders messages so they arrive in the order that the application's expecting to receive them.

The messaging layer also deals with authentication and access control on behalf of the application – which again allows the programmer to worry about the application-layer functionality and not fight with the lower-level stuff. Authentication is, of course, achieved by integrating into whatever directory service is available. The actual representation of the data is largely immaterial as far as the higher-level applications are concerned, but given the general trend in the networking industry toward XML and Web Services, this is definitely the way messaging systems are going.

Message passing software is of interest to two types of person. The most obvious is the developer, who sees the messaging software as a bunch of API calls and who uses the system as a "black box" to achieve potentially complex movements of data between disparate systems with minimal effort. The other is the system manager who is implementing message passing as the underlying engine and who only ever sees the system from the high level, where it communicates with the network and the directory service, and where it provides him with statistical and maintenance information via reporting and monitoring screens.

Although many of us have never had reason to look at messaging software, the world is most definitely moving away from stand-alone processing and toward distributed enterprise software. Many of us can, therefore, expect to be working with message queueing systems sooner rather than later.