As any email administrator will tell you, managing the content of a constant flood of incoming and outgoing messages is only half the battle.

The sheer volume of email, coupled with the increasing size of files that users want to send, also puts a staggering load on the network and servers. At the same time, end users are resisting attempts to limit the size of their in-boxes, demanding the right to stash gigabytes of mail and attachments on the server.

IT clearly needs a strategy to manage the email flood.

The first step is deploying a good email monitoring and reporting tool, advises Scott Bueffel, messaging administrator at supply chain services company Con-way. He persuaded management to let him buy Quest Software's Spotlight on Exchange and MessageStats reporting and monitoring tools when email performance problems kept cropping up and he couldn't pinpoint the problem with his existing tools.

"In the past, if we had a lot of message traffic on a particular day and management wanted to know where it was coming from and why, I'd have to say, 'I can't tell you. I don't have the tools to identify it,'" says Bueffel, noting that native message-tracking utilities are only useful for diagnosing problems with a single email.

Spotlight monitors email services and provides information on things such as available storage and the size of the routing queue. MessageStats enables Bueffel to run reports on a range of usage statistics.

"We use it for all manner of reporting, based on traffic, volume, growth, forecasting," says Bueffel.

According The Radicati Group, the average corporate email user sends and receives a total of 133 messages, or about 16.4MB of data, per day. IDC estimates that the size of business email sent annually worldwide will exceed 3.5 exabytes (3.5 billion gigabytes) this year, double the amount of two years ago.

The need for organisations to implement email management software -- specifically, monitoring, reporting and archiving tools -- will become more urgent, say experts, as the volume of mail puts greater stress on storage and bandwidth resources.

"The growth in email has been steady and huge, to the point that everybody is having major storage and performance problems," says Ferris Research analyst David Via.

And because users rely so heavily on their email these days, they tend not to tolerate delays in message delivery. "People expect the messages to go from here to there in a matter of seconds. They don't care how many messages, or what the architecture is like," says Bueffel. "They just expect that when they click Send, it should be there."

Monitoring for bottlenecks
Although email servers such as Exchange come equipped with utilities, they often can't provide the depth or breadth of information that email managers need in order to diagnose -- and anticipate -- problems.

"Using the native message tracking [in Exchange], I could see where some of the email was going, but I wasn't able to isolate where it was coming from," says Bueffel. "We needed a more general reporting tool."

Henry Yiin, manager of systems engineering at IXIS Capital Markets North America, relies on a Network Physics NP-2000 appliance to monitor traffic volume across the network and to and from the Exchange server. That has helped him pinpoint the source of problems -- whether it's the network, Exchange or some other application -- fairly rapidly.

But monitoring email storage has been more of a challenge. Yiin is using a home-grown Perl script to check storage levels on the Exchange server and associated SAN devices every 10 minutes. The Exchange server can, however, suddenly outstrip its available disk space and crash. "If storage gets to 99 percent capacity, the residing data stores shut down. That can happen in just a couple of minutes," Yiin says.

Since having the Perl script run more frequently would consume too much CPU time, Yiin is looking for a commercial email monitoring and reporting tool to replace it. "We want both real-time monitoring of the SAN status and good reporting," he says.

Sara Radicati, CEO of The Radicati Group, says she believes the market is ripe for better tools to configure, monitor and plan email systems. "That whole area has a lot of growth potential ahead," she says. "People need a lot of intelligence on performance, capacity planning, pinpointing potential problems before they occur."

To make it easier for IT to spot problems quickly, vendors are adding visual features such as graphics of the network and email traffic flows, she adds.

For example, Con-way's Bueffel uses the Topology Viewer feature in Spotlight on Exchange. Data collected by Spotlight -- such as the SMTP and MAPI connections, the available storage, and response time of the mailbox store -- is displayed in both a diagnostic console and graphical topology viewer. "You can see where there is a bottleneck," says Bueffel. He uses the MessageStats reporting tool to help him identify trends and forecast Con-way's future Exchange capacity needs. The reporting and modelling capabilities also help him stay ahead of demand, Bueffel says.

University Health System in Tennessee employs DYS Analytics' Control to manage its Exchange servers and diagnose problems with email delivery or configuration issues. UHS also uses Argent Software's Guardian database monitoring product to watch for bottlenecks in the email databases and queues.

"If the server stops, it will restart it for me or page me for other problems. It's like having an extra employee keeping an eye on things," says systems manager Jerry Hook.

Managing the volume
To help reduce the load on the email server, many companies move older messages and attachments off of it and onto a separate network server. Email archiving tools can make it easy for users to access archived email alongside their current in-boxes.

Nino Silvano, CIO of Argix Direct, a logistics service provider for the retail industry, uses EMC EmailXtender software to move email that is more than 30 days old to an archive that users can search. By offloading the messages from the Exchange server, users can be allotted more room for current mail -- up to 2GB per mailbox, though few ever need that much, says Silvano.

"We had a 180GB message store that we brought down to 30GB," he notes.

At UHS, email archiving on the Veritas Enterprise Vault has made a big difference in performance. "It keeps my email databases slim so they're running fast," says Hook.

He also saves space by using the DYS Control administrative software, which helps with housekeeping tasks such as cleaning out unused mailboxes. With 4,200 employees -- and frequent turnover for some positions -- abandoned mailboxes can use a lot of space.

"This gave me the reports I was looking for to isolate the mailboxes that needed to be deleted. There were about 1,000 of them," says Hook.

He also recommends configuring email servers to reduce traffic. UHS uses Control to identify the departments and users that exchange the highest volumes of email. Hook then locates those mailboxes on the same Exchange server. "That way, the mail isn't always going over your network," says Hook. The strategy also helps when the mail contains large attachments, like graphics files and PowerPoint presentations.

Large attachments have been a problem for Ray Martin, the IS technical manager at Monrovia Nursery in California. "We have had some PowerPoint presentations of 120MB that go out to a dozen people, and all the other outbound email backs up behind them," says Martin. "We've attempted to restrict the attachment sizes, but the limits just keep going up."

Shared file spaces and FTP sites are an obvious solution. But it's often very difficult to persuade employees to use those options, when clicking "attach" in their email clients is so easy to do. So Martin has turned to TrafficShaper, a tool from Packeteer that's designed to ensure that big attachments don't block the network.

Traffic prioritisation tools recognise different types of IP traffic and allot different amounts of bandwidth to each, based on organisational policies. Some time-sensitive services, such as voice over IP or virtual private networking, might guarantee that a small amount of bandwidth is always available.

"We don't want huge attachments blocking customer traffic from our Web site or employee VPN traffic from our remote locations," says Martin. "This gives us guaranteed quality of service and helps to juggle all the priorities."