Disk drives and Storage Area Networks (SANs) have long been viewed as a one-size-fits-all solution for performance and storage. But the invention of server-based NAND flash memory has changed the storage landscape, and we are seeing storage evolve into two purpose-built specialisations.
Traditional storage infrastructure like SANs are now used more often for long term retention, while new server-connected flash technologies are now the optimal choice for accelerating enterprises’ most critical applications. Both functions are critical and valuable for enterprise applications and this split is a natural result of the right application of the different technologies’ strengths.
Disks are great for capacity and retention, less so for performance
Before flash memory solutions hit the market, DRAM or racks of high-performance disk drives were the only options architects had for high-performance systems. The problem with DRAM is that it is very expensive, has limited in-server capacity and no persistence. The problem with disks is that, while they provide high capacity and persistent data retention, a typical disk drive can deliver only 100 to 200 I/O operations per section (IOPS).
Configuring disk drives for performance has been an art in the storage world that leads many to unnatural acts such as:
- Buying more capacity than needed, since IOPS are driven by the number of disk drives and not disk capacity
- Placing data on select portions of the drive platters (short stroking) so that disk drive heads do not have to move much, avoiding long seek times
- Investing in more expensive technologies like Fibre Channel or SAS, compared to lower cost SATA drives
- Purchasing very large storage arrays that can house hundreds or thousands of disk drives just to get the IOPS performance needed for specific applications
In the end, most storage administrators must maintain complicated data tiering or data management processes that balance data between fast, expensive disk configured for IOPS and slow, cheap disk configured for capacity and low cost.
Flash memory products, on the other hand, routinely deliver hundreds of thousands to millions of IOPS from a single persistent device that may not be that much bigger physically than a standard disk drive, and the usable capacity per flash memory product is on the order of terabytes. This flash capacity is much closer to disk capacity than DRAM capacity.
Server centric flash memory solutions free architects from configuring disks for performance
Server-centric flash memory solutions eliminate the need to configure disks for IOPS, enabling vastly simpler, smaller and more cost-effective performance solutions. Server-side flash memory comes in a variety of form factors, but the options that maximise performance are PCIe devices that reside directly on the server motherboard, close to the CPU.
The most effective of these are those that also remove the need for legacy SAS and SATA protocols and which deliver native PCIe performance of flash memory directly to applications. What this means for data storage architects, is that they no longer have to allocate excessive funds and budgets on the performance side of the equation.
Flexible deployment models for server-side flash
Server-side flash deployments present a range of options that fit just about every use case in the enterprise.
Select files on flash
Most enterprise databases and applications allow data administrators to specify storage volumes for certain pieces of data. For example, most databases allow for the placement of temp files, log files or index files onto a specific data store, and it is often best practices to do so. Storing these types of files on flash memory takes the performance load off existing SAN equipment.
The result is higher application performance for applications accessing data on flash memory, and higher performance for applications accessing the SAN, as they are doing so with far less resource contention. The result is far more usable capacity on the SAN, as there is less need for short-stroking, freeing capacity for storage, all of these serve to extend existing SAN life.
All data on flash
This configuration provides the most ground breaking approach by removing the need to store data on any mechanical disks. By keeping the entire dataset on flash, administrators are guaranteed maximum performance all the time. Availability requirements are met by maintaining multiple servers and replication. Keep in mind that flash-to-flash replication can be significantly faster than scenarios with disks, allowing for replicating more data in a shorter period of time. Many who choose this configuration still maintain long-term archives to hard disks, but don’t need to pay for performance disks and SANs to maintain live data for high availability.
Caching allows organisations to implement flash performance without any modification to their storage/SAN infrastructure whatsoever. This model is particularly suited for virtualisation, when multiple servers need to access a single data store for virtual machines and I/O workloads are complex and unpredictable.
When choosing flash-caching technology for this purpose, its important organisations can maintain compatibility with popular virtualisation tools like vMotion from VMware.
Flash delivers on acceleration benefits today
The impact of server-side flash can be significant. Some examples of performance improvements for the models we just described include:
- Select files on flash. One company placed frequently accessed data on flash memory to improve the performance of intensive queries accessing that data by 40x.
- All data on flash. Placing the entire database on flash memory improved the replication speeds for a company by 30x.
- Caching. Customers have seen 3x performance improvements for virtual machines, while retaining their existing SANs and vMotion capabilities.
With looming data growth rates in a real-time world, the good news is that we no longer need to configure disks for IOPS. Now with multiple deployment options to choose from, it is time to see how flash can improve your own application performance.