IT managers are "grossly underestimating" the explosion of unstructured data in the enterprise, according to Hewlett Packard.

HP published new research carried out by Coleman Parkes Research, which surveyed more than 1,020 CIOs and department heads of large enterprise organisations across the UK and Europe.

It found that on average, European companies believe than only 25 percent of their data is currently unstructured (ie emails, Word documents, third-party files). UK companies estimated that 29.4 percent of their data is in an unstructured format.

This, says HP, is in stark contrast to research from industry analysts that indicate more than 70 percent of information is actually unstructured data.

"Unstructured data is the 'dark matter' (ie the missing factor) of enterprise information," said Erik Moller, the EMEA marketing director, Information Management at HP Software. "Companies will fail to fully understand their business information by ignoring their unstructured data," he warned.

"CIOs expect a reduction in unstructured data in three years time," Moller said, speaking to journalists at a roundtable event in London earlier this week. "We have found that companies are not aware of how much unstructured data there is out there. It is a misperception by businesses of the way things are going," he added. According to Moller this unstructured data needs to be managed properly so that it could be mined for BI (business intelligence) purposes for example.

Moller also suggested the companies should be looking to try and automate their information management as much as possible.

"You can debate endlessly about information management," said Martin Atherton, research director at research house Freeform Dynamics. "But it is clear that the bottom line for using information resources in organisations is to make better decisions, faster."

"Information management is not divorced from the rest of the infrastructure," he added. "Big companies are only minutely ahead of other sized businesses in the information management stakes." He advised that for information management to be successfully realised, its needs ownership, and pointed to research conducted by Freeform Dynamics that suggests the single point of bottleneck is information classification.

"Unfortunately, there is no effective method today for information classification that allows organisations to do it on the fly, without impacting the business," said Atherton.

This thought was echoed by David Jenkins, applications manager at Wiltshire County Council. He made it clear that because of the Freedom of Information Act in the UK, it would be impossible to compile with the law unless their data was stored in electronically and in a structured format.

"Our biggest challenge was to make sure staff saved data correctly, in the correct place, and with an appropriate sell-by date on it," said Jenkins, liking it to a ‘supermarket shelf stocking plan'. Business classifications were a big challenge he said. "Getting those classifications right, then takes care of security, compliance etc," Jenkins added.

He also highlighted the cultural change that was necessary when implementing an information management strategy, as people often regard data as their own, and are used to storing it their own way.

Back in July, a survey by the Ponemon Institute warned that corporate information was in danger of being compromised, because IT governance policies and access rules in many companies were incapable of dealing with a massive growth of unstructured data.

But the good news is that the HP survey also apparently shows that organisations are embracing information management techniques, with 84 percent of European respondents saying they saw improved collaboration and less duplication, and 83 percent reporting improved customer service in response to Coleman Parkes Research asking respondents to cite which benefits they have actually seen or experienced in their companies so far over what they judged to be the benefits. A further 81 percent reported better segmentation of data for business use.