Have you ever wondered what would happen if - say - BMW and Volkswagen decided never to provide mileage information? Wouldn't car buyers wonder why and try to understand the reasons behind this decision?

The Storage Performance Council has developed a benchmark, known as SPC, a sophisticated performance measurement workload for storage subsystems, that simulates the demands placed upon on-line, non-volatile storage in a typical server class computer system. Here is a quick summary of who is providing ‘mileage' information based on this independent storage benchmark today:

(Note that SPEC-1 is for block serving and SPEC-sfs-97 is for file serving. Scroll down for the table.)

Vendor SPC-1 SPEC-sfs97
IBM Yes Yes
H-P Yes Yes
Sun Yes Yes
3PAR Yes Yes
EMC/Dell NO Yes
NetApp NO Yes

Don’t you think that each of the SPC-1 holdouts represents an interesting riddle?

- HDS: buys into the value of the SPC benchmark (it is an active SPC member), but apparently believes only others' results should be known, not its own.

- EMC: once an SPC member, it then withdrew saying SPC ‘has very little, if any, relevance to overall system performance and the value customers receive from EMC Symmetrix systems.’ This begs the question: why isn't the same true of SPEC-sfs97 with respect to EMC's Celerra systems? And what about CX systems?

- NetApp: believes in the value of independent block benchmarks (here is the link to its commissioned benchmark against EMC’s CX500:- http://www.netapp.com/ftp/veritest-netapp-comp-analysis2005.pdf), but perhaps just not the SPC-1 variety. Its only known statement regarding SPC (from its white paper TR-3239) suggests NetApp feels the SPC-1 may be too much effort: ‘Formal benchmarks such as SPC-1 […] require the publication of dozens of pages of charts and graphics to describe the system's performance.’ Ironically, its own 37-page benchmark report is full of charts and graphs.

If you suspect that these companies withhold SPC "mileage" information because they believe it could harm their businesses, consider what is at stake: approximately 55 percent of the industry's revenues might be less certain if customers knew the truth. Here is how this figure is calculated.

Below are the top seven worldwide players in Controller-Based Disk Storage (Gartner numbers). They comprised almost 80 percent of the market in 2004, or $10.5 billion. Since Sun and HP OEM from Hitachi, a simple 50 percent revenue split is assumed:

(Scroll down for the table.)

Rank Vendor Rev.share Revenue ($M) SPC-1
1 EMC 23.1% 3,104.90 NO
2 HP - HDS est. 8.3% 1,120.10 NO
2 HP - non-HDS est. 8.3% 1,120.10 Yes
3 IBM 12.4% 1,663.30 Yes
4 HDS 8.8% 1,184.50 NO
5 Sun - HDS est. 3.1% 416.00 NO
5 Sun - non-HDS est. 3.1% 416.00 Yes
6 Dell 5.8% 782.80 NO
7 NetApp 5.5% 736.60 NO
Sum SPC-1 NOs 54.6% 7,344.90 NO

The SPC-1 marketing budget, if there is one, is no match for the $7.344 billion of suppliers' revenue that may wish to diminish its influence.

Even so, some customers are well aware of the SPC - Fidelity Investments, for example, which is an SPC member.

So if you would like to find out more why not get in touch with these vendors and ask them why independent benchmarks are not their cup of tea?