The relational database may may never die, at least not anytime soon, but its days of glory appear to be over.

Relational databases, long a critical piece of enterprise software deployments, are now forced to share the stage with technologies better geared to accommodate newer data structures and modern hardware systems. Stalwart RDBMSes remain in place from software vendors such as IBM, Microsoft, and Oracle, all of which will continue to dominate such core functions as financial transactions. But NoSQL databases, as well as big data technologies such as Apache Hadoop and MapReduce, are where the action is.

"The relational database as it stands is dead," says analyst Robin Bloor, chief analyst at the Bloor Group. "Its architecture is old, and it needs to be renewed."

Bloor reasons that RDBMSes were written for old hardware environments with single CPU systems, a small amount of memory and large stores of disk space. With the growing prominence of multi-CPU computers and solid state disks, disk access is no longer as important.

"The whole thing is changed. You're going from a train to an airplane," says Bloor. Solid state disks are faster, so the ratio between disk and memory in terms of read speed will come down, he said.

The RDBMS has been devalued, says Jill McRae, a learning architect for business and intelligence systems at consulting firm Wright Robbins. "It never was completely there," she says, recalling statistics showing very little of the world's data is actually managed in relational systems.

The rise of NoSQL and "NewSQL"

McRae cites the emergence of NoSQL databases as "changing the scope of what a piece of data is." It's no longer primitive data types such as integers and floating points, data could be a whole document. "[NoSQL] probably scares the hell out of DBAs because they're losing hold of their domain."

NoSQL features databases that are non-relational, horizontally scalable, distributed and open source. They can serve as a backing store for web application servers, content management systems, structured event logging, server-side storage for mobile applications and document storage, says Dwight Merriman, coauthor of the MongoDB NoSQL database.

Database pioneer Michael Stonebraker, the main architect of the Ingres RDBMS and now CTO of VoltDB, decries legacy "old SQL" RDBMS systems. "They are all running very, very old code lines at this point," he says. "Oracle doesn't scale because of legacy problems."

He is backing "NewSQL," which he says preserves SQL and the relational model as well as ACID (atomicity, consistency, isolation and durability) while offering performance and scalability. NewSQL eliminates the resource-consuming buffer pool by running the database in main memory and removes the need for latching by running on a single thread on a server.

Also joining the fray is Hadoop, which provides for distributed processing of large data sets across computer clusters. It can scale to thousands of machines. It is complemented by the MapReduce programming model and framework for building applications to quickly process large amounts of data in parallel or in clusters.

RDBMSes handle only about 16 percent of data

Although relational systems are feeling the heat from newer technologies, RDBMS systems remain a force in enterprise computing. The RDBMS market accounts for about $35 billion, which includes software licences, technical support, maintenance and services, says Forrester analyst Noel Yuhanna. Forrester estimates that 25 percent of business data in enterprises is structured data, of which at least 65 percent is in an RDBMS or other traditional DBMS (so RDBMSes handle at least 16 percent of business data); the remainder is typically in files and text formats.

The remaining 75 percent of business data is a combination of semistructured documents (such as XML, emails and EDI) and unstructured data (such as documents, pictures, audio and video). "We estimate about 5 percent of this data resides in RDBMS, the others are in different non-RDBMS and file formats," Yuhanna says.

The RDMBS, Bloor says, "could die without anybody noticing it." Oracle, for example, could acquire newer database technology and switch out its old engine. "It would still be named Oracle, but it wouldn't be Oracle anymore."

A column store database could be a candidate for such an transitional acquisition, he says. Or a relational database product simply could be rewritten from scratch. But if IBM, Microsoft and Oracle have any plans to modernise their RDBMS, they're not saying.