Oracle's powerful new HP Oracle Database Machine comes with 168TB of storage, a new method of retrieving data more quickly and intelligently, and - gulp- a  hefty $2.33m (£1.26m) price tag.

It's the turbocharged option for the database administrator with money to burn and a need for speed.

But most DBAs don't get to drive in the fast lane - especially not with IT budgets the way they are. So as a less lavish option for enterprise users, Oracle is touting another approach. That one involves data compression, which has long been a popular way to save storage space and money. Traditionally, though, the trade-off has been high: Gobs of memory and processing power typically are needed to compress data and write it to disks. Even more is needed when the information is later extracted.

Now Oracle claims to have solved this thorny problem with a feature it first introduced in its Oracle 11g database, which was released last year.

By using the Advanced Compression option in 11g, Oracle says, DBAs can shrink database sizes by as much as three-quarters and boost read/write speeds by three to four times, no matter whether they're running a data warehouse or a transaction-processing database - all while incurring little in the way of processor utilisation penalties.

Oracle claims the storage and speed gains are so dramatic that companies using Advanced Compression will no longer need to move old, seldom or non-used data to archives. Instead, they can keep it all in the same production database, even as the amount of data stored there grows into the hundreds of terabytes or even the petabyte range.

"This works completely transparently to your applications," according to Juan Loaiza, Oracle's senior vice president of systems technologies. "It increases CPU usage by just 5 percent, while cutting your [database] table sizes by half."

Oracle says it's responding to the demands of enterprise customers with fast-growing databases. "The envelope is always being pushed," Loaiza said. "Unstructured data is growing very quickly. We expect someone to be running a one-petabyte, 1,000-CPU-core database by 2010."

It's also responding to the fact that storage technology, one of the keys to database performance, has made little progress from a speed standpoint, according to Loaiza. "Disks are getting bigger, but they're not getting a whole lot faster," he said.

Taking data compression down to the block level
Oracle has offered simple index-level compression since the 8i version of its database was introduced in 1999. That improved several years later with the introduction of table-level compression in Oracle 9i Release 2, which helped data warehousing users compress data for faster bulk loads, according to Sushil Kumar, senior director of product management for database manageability, high availability and performance at Oracle.

Advanced Compression provides even finer capabilities, letting the database compress data down to the disk-block level. The algorithm used in the new feature compresses data while keeping track of exactly where information is stored, Kumar said. The result, he claimed, is that when data is extracted by users, the database can focus in like a laser on the exact block on the disk where the information is located, instead of pulling whole tables and sifting through unwanted data.

Other compression schemes "have no idea what's on the disk," Kumar contended. "They can't read part of a document without opening up the entire one."

According to Oracle officials, Advanced Compression is also smart enough not to compress data with every single change to a database, but to instead let the changes accumulate and then run them in batches. That is efficient enough to enable Advanced Compression to work with OLTP databases, which tend to have heavy read/write volumes, said Vineet Marwah, a principal member of the Oracle database staff.

Another component of Advanced Compression, called SecureFiles, can automatically detect, index and compress non-relational data such as Word documents, PDFs or XML files, Marwah said. Oracle also has enhanced its backup compression performance so that it is 40 percent faster in 11g than in the previous version of the database, while not degrading the performance of other database functions, he said.

And because a compressed database is generally much smaller, it shrinks the flow of data between the storage server and database, where bottlenecks tend to occur, Kumar says. The gains are so dramatic that DBAs can dump their complicated partitioning and archiving schemes, he claimed. "A lot of people archive data because they have to, not because they want to," he says. "So if you see a business value in keeping data around, compression is a useful way to not let resource constraints dictate your architecture."