A couple of weeks ago, Seagate announced and demonstrated that it was capable of doing native command queuing (NCQ) on Serial ATA (SATA) drives.

This feature was previously only being demonstrated as a technology preview but is now being included in Seagate Barracuda SATA drives. While NCQ was originally seen as an extension to SATA 1.0 it is a key element of the Serial ATA II standard.

For most people this probably didn’t mean anything but for the storage community and especially the Network Attached Storage (NAS) vendors, this is big news. It also changes the way that disk subsystems are likely to be deployed over the next year.

Parallel ATA (PATA) only deals with commands on a first come, first served basis. If you have a primary and a secondary disk and send commands to both, the interface will only allow one drive to respond until the command is complete. ATA acts, therefore, as a blocking mechanism.

With SATA NCQ Seagate has moved SATA drives away from the pure personal drive market and into the server market. This move had already begun to happen with an increasing number of low-end servers already sporting high capacity PATA drives. The problem with PATA drives is that they lack performance. SATA drives are already being deployed in some mid-range systems and NAS installations but NCQ means that many people are likely to see them as a preferred solution to SCSI, in all bar critical systems.

SATA NCQ allows a drive to use internal memory and take up to 32 different commands and re-order them to get the best performance. Much of the performance gain of hard disks over the last decade has come through algorithms that reduce the amount of disk rotations and head movements to position the head over the location where data is to be read/written. Simply re-ordering the commands will not necessarily improve the performance of a drive.

What SATA NCQ does is to add a number of additional elements to the management of the command queue. There are three key elements. The first is known as the Race-Free Status Return Mechanism (RFSRM). RFSRM allows a drive to communicate the status of any command in real-time rather than create a handshake to talk to the host system and report the status of any of the commands.

The second element is Interrupt Aggregation. Whenever a command is completed an interrupt is generated so that the host knows the status of a command and queue. With disk-intensive activities the number of interrupts can significantly affect systems performance. This is typical of web-based activities where much of the data to be read/written is in very small files. NCQ allows these interrupts to be aggregated, therefore it is possible for an entire queue to generate just one interrupt even though multiple read/writes may have occurred. This is based on the time taken to complete the task.

The third new element is First Party Direct Memory Access (FPDMA). This allows the drive to manage data transfer without host software intervention. The host controller is able to load a particular command into the DMA engine causing the data transfer to proceed without software intervention. With FPDMA, the drive can now manage its own command queue by selecting the buffer to transfer.

The big gain from FPDMA is that the drive itself knows exactly where it is relative to the data on the disk. This location information combined with FPDMA means that the most effective match can be made between the command queue and the head position.

One of the bugbears of any disk subsystem is data fragmentation. When data has to be retrieved from a badly fragmented disk, performance can be severely degraded. This is because the drive may have to make many additional rotations of the disk and movements of the head to locate the data. With FPDMA, data is returned out of sequence because the goal is the most efficient pass of the heads across the disk not the need to read/write all the data in a single coherent block.

Will I see any difference?
The answer is maybe and it is application dependent.

Most applications issue commands using Synchronous IO. That means that until they get the first piece of data, they won’t ask for the second piece. In order to take advantage of SATA NCQ, applications need to simply push a “queue” of commands to the drive. If you are looking to install SATA drives with NCQ into a desktop or workstation then you may see no real benefit until the software is written to use Asynchronous IO.

For servers, the story is different. Current server operating systems and applications already provide support for command queuing, especially in the enterprise space where SCSI drives have been dominant.

Hardware changes will also improve the changes of better support for SATA NCQ. The latest families of processors support Hyper-Threading technology. Hyper-Threading allows for queues to be built even if the application only uses Synchronous IO.

SATA NCQ is undoubtedly a significant advance in drive technology. The combination of command queuing for system performance and low-cost drives should have a significant impact on the cost-sensitive server market. It will also provide NAS vendors with technology that will seriously advance their market.

However, it is important that SATA NCQ is not seen as a direct replacement for SCSI throughout the organisation as the drive architectures are very different. SATA drives have a single processor and lack many of the other hardware refinements of SCSI.