Iraq isn't the only war costing billions. According to Gartner, 4 percent of IT budgets now go to security hardware and software as companies deploy an army of firewalls, intrusion-detection and -prevention systems, antivirus tools and VPNs, as well as authentication, access-control and identity management systems, to keep out hackers, criminals and other marauders.

But as any commander will tell you, battlefield success isn't just a matter of superior firepower. It also depends on communication.

From an IT standpoint, the challenge is how to best turn a ragtag bunch of security assets - each with its own log files, its own data structure and its own rules - into an elite defensive unit. To achieve this, companies are turning to security information management (SIM) software, which is designed to do for security what products such as Tivoli have done for networks - simplify management, provide greater visibility and improve response times.

Chaim Feldman, network and systems security manager at Bezeq - The Israel Telecommunications, Israel's national telecommunications provider, reports that his company wards off 3,000 to 4,000 attacks per month. To gain control over the situation, he installed Computer Associates International’s eTrust Security Command Center (SCC). "Before I had this, I was kind of blind," he says. "I couldn't actually see all of my company."

Stopping the data deluge
Establishing comprehensive security requires more than just deploying an ever-expanding array of software and devices. It also means turning them into a coordinated set of managed assets.

"Everything was a big mess," says Jim Patterson, security analyst for the Illinois state government's Legislative Information System. "Even from a major vendor like Cisco, each device had its own reporting console." To make matters worse, Patterson found that each device's software package typically had to run on its own PC. That meant having to track alerts from an array of machines, all of which needed to be logged onto separately.

Further complicating the issue, the firewalls generated more log traffic than the built-in database could handle. At the recommendation of Cisco Systems, Patterson installed nFX Open Security Platform from netForensics. "With SIM in place, you can reduce the number of people you need to have monitoring things, since everything is coming into one central station," says Patterson.

Security information management (also called security event management, or SEM) is an outgrowth of the event logs that managers used for network management but is tailored to gather data from security devices.

"The big differentiator is that, in addition to all the logs, a SIM can intake data from devices that don't generate logs or that generate robust but very specific proprietary information," says Gartner analyst Amrit Williams. One of the key functions is to reduce the number of false or inconsequential alerts that employees must review.

"We began implementing intrusion-detection systems a few years ago," says Chris Rein, chief of infrastructure and production services for the New Jersey Office of Information Technology. "The volume of data would overwhelm any person or small group looking at it."

Rein's experience is far from unique. Ulises Castillo, CEO of Scitum SA, a managed security service provider in Mexico City, says that he feeds 3 million to 10 million events per day from the 350-plus devices he manages into his Security Threat Manager 3 SIM from OpenService in Westboro, Mass. In doing so, he reduces the number of alerts by a factor of 10,000.

Dan Lukas, lead security architect at Aurora Health Care, a Milwaukee-based provider with 30,000 employees at 13 major hospitals and several hundred clinics and pharmacies, says that depending on the time of day, his company receives as many as 5,000 to 10,000 events per second. He uses Intellitactics Network Security Manager from Intellitactics, to visualize the source of problems.

"We can play events back and see what devices it is hitting and track it back," says Lukas. "We have quite a few thousand switches out there, but we are getting to the point where we can see which port in the entire network a situation is coming from."

Companies looking to install SIM systems have a choice between server-based software and appliances. The New Jersey government chose the latter option. It uses PN-MARS appliances from Protego Networks, which was recently acquired by Cisco. Rein says the state started with a few appliances for testing and is now purchasing more to put its WAN under complete surveillance. Beyond that, it will enter into agreements with agencies to set up more specific and granular monitoring of internal networks.

"Staff feels strongly that it has helped them improve their productivity level, since they don't have to mitigate as many issues as they experienced before," says Anna Thomas, New Jersey's chief of strategic development and digital communications. Bezeq took the software approach. The company runs CA's eTrust SCC on a server sitting on a separate security network that monitors two production networks - one for customer services and another for internal operations.

Feldman says that having the SCC has given him better visibility into the types of attacks on his network and where they are hitting. For example, he once spotted a recently fired employee attempting to gain access to a sensitive server. Another time, the data-correlation feature detected a virus starting to spread, allowing Feldman's staff to disconnect the subnet to stop it.

Setting expectations
Implementing a SIM is a major undertaking. Installing the software or appliance itself is generally simple enough, but setting up the desired monitoring and reducing the number of false positives takes work.

"When (companies) embark on a large SIM project - about 300-plus audit sources/nodes - they should put aside at least US$50,000 in services budget for the vendor or a competent third party to come in, install and tune for appropriate business requirements," says Paul Proctor, an analyst at Meta Group Inc. "Deployments involving over 1,000 monitored nodes are usually multiyear efforts, so set realistic expectations and project goals."

John Summers, Unisys Corp.'s global director for managed security services, is engaged in just such a large-scale project. He's installing ArcSight software at his company's three security operations centres, which manage security for 200 customers in addition to handling Unisys' own needs. The installation began in June 2003, and Summers expects to be able to view security events on a global scale sometime in the first half of this year. He can already manage customers on an individual or regional basis and has been able to detect zero-day attacks on one customer and harden the defences of other customers before they're hit. "The only way to do that is to have a platform that can do complex pattern detection across a heterogeneous infrastructure and across time," says Summers.

Michael Gabriel, corporate IT security manager at Career Education Corp (CEC) in Hoffman Estates, Ill., recently installed netForensics software to meet the auditing requirements of the Sarbanes-Oxley Act. "It's a bit of a challenging environment, since we run a number of separate Active Directory forests," he says. "We have a collector device in each AD forest, which collects the events from the Windows Domain Controllers, firewalls and IDSs and sends these to a centralized netForensics collector."

To meet the regulatory deadline, CEC purchased five days of consulting time to quickly get the netForensics software up and running. That was enough to establish compliance, but Gabriel says he still has a lot of work to do to fully use the system's functions. "It's a complex system, and you will get out of it what you put into it," he says. "I still feel we have just scratched the surface of its capabilities."

Part of what companies must put in is the work to ensure data quality. "SIM suffers from a huge garbage in/garbage out problem, and most of the data collected by enterprises is garbage," says Meta Group's Proctor. "They don't usually realize this until after spending several hundred thousand dollars on a huge monitoring infrastructure only to realize they are deriving no benefit."

Like other types of management software, SIM is only a tool. It can assist your security personnel in better securing the enterprise, but it's not a replacement for their perception and skills.

"You automate data collection, data correlation, data search - the tasks that are boring and suitable for computers," says Bruce Schneier, an author and chief technology officer at Counterpane Internet Security. "But you can't automate intelligence, and you can't automate creative thinking."

The market and the players
According to Gartner, the worldwide SIM market had grown to $100 million by 2003. The Stamford, Conn.-based research firm also estimated that 20 percent of Fortune 1,000 companies were using security management software in that year and that vendors would double their installed bases in 2004.

The SIM market is divided into two main categories. First, there are vendors that specialize in SIM, such as OpenService, netForensics, Intellitactics, ArcSight and Network Intelligence. Then there are larger companies that sell SIM products to complement their other management or security offerings. They include Computer Associates, IBM's Tivoli Software unit, Symantec and NetIQ .

The smaller vendors' products generally offer more complete sets of features. But the larger companies offer products that are interoperable with their other offerings and that they can sell to their existing clientele.