Companies looking to clamp down on data leaks may be introducing a whole new set of security problems, researchers from Matasano Security have claimed.

Matasano has spent the past nine months testing a range of information protection products for bugs, on behalf of corporate customers that were looking to deploy the systems, Thomas Ptacek, a researcher with the company, said at the Black Hat conference. Ptacek's company tests commercial products and generally expects to find vulnerabilities whenever it engages in this type of testing, Ptacek said. "In this case we were not disappointed."

The researchers focused on products that install "agent" software on desktop PCs in order to monitor things like web browsers, email and instant-message conversations, looking for data that might be leaving the corporate network.

These agent-based products all suffered from a similar set of problems, said Eric Monti, lead security consultant with Matasano. "There's too much trust placed on the agent, and you eventually have to think of that agent as a potentially malicious enemy," he said in an interview after the talk.

The researchers found a number of flaws with the software they examined. For example, they were able to exploit a bug in the way agent software parsed AOL's instant messaging protocol to seize control of an agent computer. They could overwrite events logs in the management console, and they found that clients reported data to management servers in an unencrypted format.

Agent-based data-leak prevention products, which are sold by companies such as WebSense, Verdasys, and McAfee are generally considered to be more effective than products that look only at network traffic, said Andrew Jaquith, an analyst with Yankee Group.

He was not surprised that Matasano found security holes in this type of software. "Most security vendors write code that isn't that great from a security perspective," he said. "They tend to be smaller companies with smaller development staff. They're putting in features as fast as they can and they're not going to have the deep pockets of a Microsoft."

But because these systems are not widely deployed, they are unlikely to be exploited by attackers, Jaquith added. "If I was writing malware, I would write for one of the anti-virus vendors, just because they're widespread."

Matasano examined McAfee's Onigma data-loss prevention software, among others, but the researchers didn't uncover anything that needed to be fixed, said John Viega, vice president and chief security architect with McAfee.

"Basically they were saying that with software that has a presence in the kernel, you should be security conscious about it and should go to the vendor and ask them questions," Viega said. "There is no undue reason to be concerned, at least in our case."

Matasano didn't back up its claims with any bug disclosures. The company declined to provide details on any of the flaws because the vendors had not patched them. In fact, Matasano declined to even name the products it evaluated, saying only that it had looked at between four and eight products.

"We had hoped by the time we gave this talk that a lot of the vulnerabilities we were going to talk about would be old news ... that's not the case."

Originally scheduled for Wednesday, Ptacek and Monti's presentation had been delayed and almost didn't happen. Sources familiar with the situation said that an unnamed vendor pressured Matasano into not presenting the talk, which was pushed back to give the company time to resolve any possible legal issues.

Instead of discussing specific vulnerabilities, the researchers said that their talk was intended to give IT workers a list of questions to ask vendors before rolling out information protection systems.

"These vendors... are very good at selling to C-level executives [telling them] that they will keep them out of the newspaper," Ptacek said. "The real issue here is that extrusion prevention systems are a top-three budget item and you're going to wind up deploying these systems."