According to IDC’s latest server market share report, released earlier this week, Oracle is languishing in fourth place with 6.6 points of market share (IBM has 30.5 points of market share). When you consider that Oracle/Sun has lost server market share in each of the past seven quarters, you have to imagine that Oracle are desperate to stop the rot. Well, today our friends at Redwood Shores attempted to stem the tide by announcing a new TPC-C benchmark result for a cluster of SPARC systems. However, the benchmark result is far from impressive. Sure, the benchmark system has a huge throughput. However, it is woefully inefficient.
Today’s Oracle benchmark result uses 27 64-core Sun SPARC T3-4 servers to process more than 30M tpmC*. In contrast, IBM’s most recent clustered TPC-C result uses 3 64-core IBM Power 780 servers to process more than 10M tpmC**. There are many ways to look at this. You could claim that Oracle uses nine times the number of CPU cores to achieve only three time the performance. Alternatively, you could claim that each CPU core of the IBM system is able to achieve three times the performance of a CPU core in the Oracle system. Either way, in my opinion, it points to a very inefficient benchmark run. Such inefficiencies are surely a concern for customers who are paying for Oracle Database based upon the number of CPU cores in their systems.
At first glance, the cost efficiency of this new benchmark system from Oracle may appear to be impressive—their system costs 1.01 USD per tpmC. However, if you scratch below the surface, you will find that number is quite deceptive. Oracle do not use the perpetual licenses that you would expect, and Oracle do not use the kinds of support contracts that you would expect. If they did use the licenses and support contracts that are most commonly used, then the system costs would skyrocket, and the relative cost inefficiencies of this system would be plain for all to see. For prior coverage of Oracle’s price/performance tactics, see Sun and Oracle TPC Price/Performance Tactics Revealed.
Also, you should be aware that Oracle have once again resorted to sacrificing data integrity for performance in its benchmark systems. They have turned off page integrity checking—I imagine because, according to the Oracle documentation, it incurs a performance degradation of between 1% and 10%. So, even though it is highly unlikely that you would run a production system without page integrity checking, Oracle has chosen to do just that in the interests of squeezing extra performance out of its system.
Given all this context of misleading cost information and questionable system settings, it was timely to read the following article yesterday… Larry Ellison Hearsay: “We Can’t Be Successful if We Don’t Lie to Customers”
Results on Transaction Processing Performance Council Web site at www.tpc.org. Results as of 12/02/10.
* Oracle SPARC SuperCluster with T3-4 Servers (27 x 64 core) (108 chips, 1728 cores, 13824 threads); 30,249,688 tpmC; $1.01/tpmC; available 6/1/11.
** IBM Power 780 cluster (3 x 64 core) (24 chips, 192 cores, 768 threads); 10,366,254 tpmC; $1.38/tpmC; available 10/13/10
Just build something that achieves 30m tpmC and costs less and this talk is over. These benchmarks are just technology shows, customers want a complete solution that works (including the business side as IBM likes to do it).
Pingback: Oracle and the deceptive art of benchmarking « Michael Curry: Information Explosion
“Given all this context of misleading cost information and questionable system settings, it was timely to read the following article yesterday” … which references a 30 year old sentence …
By the way .. there are rules to get a TPC-C Benchmark result. You can say this rules are stupid and this Benchmark says nothing for practice use (like a formula one car had nothing do to with a buyable car). But to publish own benchmarks and than to badmouth better results from competitors is not the behavior of a sportsman …
The great thing about these industry benchmarks is that we do get to see all the details… the exact hardware used, the exact sofware used, the pricing details, even the software configuration settings. This provides a welcome level of transparency into exactly how a vendor achieves its result. When a vendor like Oracle publicly claims to be less costly than IBM, yet they fail to mention that they are comparing a 3-year term software license for the Oracle result to a perpetual software license for the IBM result, I think its fair to point it out. And, of course, I think its also fair to point out what Oracle is doing regarding support and maintenance in these benchmark results. People deserve to know. When Larry Ellison goes on stage and publicly makes brash statements that are misleading, I don’t think it is unreasonable for someone to later document the real story.
Well, TPC benchmarks need define rules for cost calculation to avoid further misleading results.