The EMC bloggers have apparently received their orders to spread some FUD about standardized benchmarks – first storage oligarchist and then, so predictably, that other guy who isn’t even worth a link. It’s the most egregious example I’ve seen lately of argument by demanding impossible perfection. Are SPC benchmarks perfect predictors of customer results? Of course not. Are they better than nothing at all? Definitely. “Not perfect” and “worthless” are two distinct concepts. All of the bluster about “real work” and “generating facts customers can use” is pure hot air. What is EMC proposing as an alternative? Nothing, really, except for other vendors to take all of EMC’s claims at face value and never make counterclaims of their own.

EMC knows that bringing vendors in to tune and test on real applications is a pretty large investment for customers to make. They know that if customers have no other criteria to go on, they’ll only invite the industry incumbents. By casting aspersions on the very notion of standardized, repeatable, auditable benchmarks they hope to ensure that no customer will use such benchmarks to generate a candidate list that excludes them, or even to find out about other possibilities. Those who fear that competitors are gaining always seek to narrow the field or end the race early, and this is just one example of how they do it.