Shared storage was once a simple animal. The abstraction between servers and storage made it possible for the server to use storage without any direct knowledge of exactly what it was or how it was configured. This was a great thing for the industry because it allowed storage interconnect standards such as Fibre Channel and iSCSI to allow interoperability between just about any compliant server operating system and storage array.

All that is starting to change. Over time, storage vendors have come out with proprietary, server-based software to allow servers to work more intelligently with storage. At first, it was common to see multipath IO DSMs (device-specific modules) such as EMC's PowerPath. These packages allowed the server to intelligently balance the flow of data over multiple physical paths to the storage device, direct traffic to the most advantageous storage controller port, and transparently work around fabric failures.

Today, these software offerings have become much deeper and more feature-rich. Most of the major storage vendors now supply software that makes it possible to take application-aware SAN snapshots. These packages allow the application, usually database platforms such as Oracle or Microsoft SQL, to make sure their data is in a consistent state before the SAN creates its snapshot. In many cases, these snapshots can be simultaneously leveraged by backup software to make application-consistent direct from SAN backups.

That's by no means the end of the story. For example, a few weeks ago Dell announced the availability of an early production release of the new 5.0.0 firmware for its EqualLogic PeerStorage array family. Among the many improvements in array and management interface functionality, this release introduces support for VMware's vStorage APIs for Array Integration (dubbed VAAI).

VMware's vStorage APIs for its popular vSphere hypervisor were first announced nearly two years ago at VMworld 2008. The vStorage marketing moniker introduced such features as APIs that cover storage-aware multipathing, data protection, and site failover. Since 2008, all those features have been implemented and released in one form or another. The array integration piece is due to be supported in VMware vSphere 4.1, which we're likely to see later this year (probably timed to coincide with VMworld 2010).

VAAI brings platform and storage integration to a new level. Cloning virtual machines and moving them from one SAN volume to another are common administrative tasks. Currently, copying or moving a VM around on a SAN requires the vSphere host to perform the copy itself - essentially reading data off of the SAN and simultaneously writing it back to the same SAN in a different place. This process is time consuming and enormously inefficient.

What VAAI promises to do is to offload these tasks from the vSphere host to the storage itself. Instead of performing the copy or snapshot operation itself, the hypervisor will effectively tell the SAN what it needs done and the SAN will do the work internally without needing to load the host or the SAN fabric. Testing performed by some storage vendors working with beta copies of vSphere 4.1 have shown that this can decrease the amount of time it takes to complete such tasks by as much as 75 percent.

VMware's VAAI is only one example of this kind of intelligent storage integration. We will surely see more and the effect will be profound. The value that modern primary storage delivers has been shifting from raw performance and reliability - previously the defining characteristics of SAN technology - toward more advanced controller-based features such as snapshots and replication. Advanced application integration will certainly play a huge role as well.

But with this advanced functionality comes advanced complexity. As I've mentioned before, the greater the number of software pieces, the greater the potential for bugs and compatibility issues.

Instead of simply scheduling a little downtime to implement some new SAN firmware, for example, you may also need to upgrade all of the other bits and pieces of storage integration software scattered across your network - multiplying the effort required to perform what would have otherwise been a simple upgrade. Worse still, the more software that's involved, the more likely you are to trip across a potentially debilitating bug.

None of those risks should cause you to avoid this kind of technology, but it's definitely important to go forward with your eyes open. The next few years will see many further advances in SAN, platform, and application integration as vendors attempt to provide unique value. Weigh your options and stay realistic, and you'll reap real benefit.