Sun and Oracle TPC Price/Performance Tactics Revealed
Sun and Oracle have a TPC-C benchmark result that delivers 7,646,486 tpmC with a price performance of $2.36 USD/tpmC*. Here are a couple of things about that result (and other recent results from Oracle) that you may not be aware of:
- The Oracle benchmark result does not use perpetual software licenses
- The Oracle benchmark result uses a Web-based incident support contract
Oracle are comparing their result to the IBM TPC-C result with $2.81 USD/tpmC**. However, this may not be an apples-to-apples comparison because the IBM result includes pricing for 24×7 support, upgrade protection, and perpetual licenses; the Oracle result does not include any of these features. If you include 24×7 support, upgrade protection, and perpetual licensing, you’ll find that the prices most customers will pay are significantly different than what Oracle includes in the benchmark. Let’s see why this is so…
When Oracle prepares TPC-C benchmark results, they typically use a special license called the Oracle Term License (denoted by the ‘Unlimited Users for 3 Years’ text below):
The Oracle Term License is not a perpetual license, like the software license that organizations typically purchase. Instead, it is like a lease. This term license for three years costs 45% less than a perpetual license. After the three year period, you no longer own the right to use the software. At that stage, to keep using the software, you either have to purchase the software license for an additional term or purchase a perpetual license. To truely compare this result with TPC-C results that use perpetual licenses, like the IBM results, you need to do some math with the Oracle Database license costs.
When it comes to support, things get a little more interesting. First of all, you should note that the cost of support for a term license is the same as the cost of support for a perpetual license. If you have a look at the Oracle Web site and do some math, the costs for support work out to be more than 40% of the term license cost. But, support costs for this benchmark are not more than 40% of the software license costs. They are actually a little more than 1% of the software license costs.
You see, for this benchmark and many others, Oracle uses something called the Oracle Incident Server Support Package (OISP). The OISP is a support package that has no telephone support. Instead, it allows you up to 10 Web-based incident requests per server (that expire within one year). What’s more, OISP has no upgrade protection and does not entitle you to future upgrades of the Oracle Database software. The cost of the OISP is $2,300 per server, which is why you see the cost of Oracle support for this benchmark at $82,800 (12 nodes in the cluster for 3 years of support). This represents a little more than 1% of the costs of the term license costs.
As you can see, Oracle manage to significantly improve their price performance result by using term licenses and by using a limited support offering. In the event that you do not use term licenses or this limited support offering, you will need to do additional work to see what the systems would cost for you, or to compare the Oracle results with other TPC-C results.
All TPC results available on the Transaction Processing Performance Council Web site at http://www.tpc.org.
* 12-Node Sun SPARC Enteprise T5440 server cluster; 7,646,486 tmpC; $2.36/tpmC; available 03/19/10.
** IBM Power 595 Server Model 9119-FHA; 6,085,166 tpmC; $2.81/tpmC; available 12/10/08.
So that’s your rant?.. Oracle’s solution still provides better performance, needs less physical space, and less cooling requirements than IBM’s. And your only complain is perpetual licenses vs subscriptions?
Phobos
October 24, 2009 at 11:40 am
Hi Phobos,
I just want to make sure that readers are aware that comparing the two “price performance” results mentioned above isn’t an apples-to-apples comparison. It is possible that people may look at the results and have the initial impression that one is better value for money than the other, but once you look at the details, you realize that DB2 is actually better value for money.
The use of flash drives plays a big part in Oracle’s performance and power efficiency claims. However, you should be aware that when the IBM configuration was created (well over a year ago), this technology wasn’t as available as it is today. Check out the following article from Timothy Prickett Morgan of the Channel Register: Ellison whips out his Sparc TPC-C test. In this article, he says “Oracle was quick to point out that response times were 16 times faster on the Oracle/Sun setup than on the IBM box. Well, no kidding. No one had enterprise-class flash drives in the summer of 2008, when IBM was putting together its test, so this is an apples-to-pears comparison.“.
Conor O'Mahony
October 25, 2009 at 8:43 am
Conor, believe me when I tell you Timothy (TPM) is not a good source for Sun, Oracle or IBM related information. His opinion is always biased against Oracle and Sun and always pro IBM, so he will ALWAYS rant against Sun and Oracle, no matter how good whatever they do is. He even censors my comments on TheReg when he is proven wrong, so don’t take his word on anything.
I know you too are pro DB2, but at least be objective and give credit where it’s due. price/perf. is just one of the points the Sun/Oracle solution had in favor, it’s not the only one, and if you want apple to apple comparison, then talk about the other strong points it has against IBM.
Phobos
October 31, 2009 at 12:54 pm
Hi Phobos,
Yes, I am obviously pro-DB2 and IBM. But I do try to be as open minded as I can be. Given what I know about our internal benchmarks with DB2 pureScale (a shared-data cluster), I was actually expecting Oracle and Sun’s RAC-based benchmark result to be much stronger. And that’s the honest truth. I guess the infamous Oracle advertisement in the Wall Street Journal had amplified those expectations. But when the result came out, I was surprised because I believe the results are not very impressive. Let’s ignore the “games” that cloud the price/preformance result for now.
My initial surprise was that the Oracle result was not run on Exadata v2. Oracle had made such a big deal about Exadata v2, but they don’t then actually use it for the benchmark. I was wondering why this is so. But that’s not directly relevant to this discussion.
What is directly relevant is the scale-out efficiency. This result is run on a cluster of 12 Sun SPARC Enterprise T5440 servers, each running 4 1.6 GHz UltraSPARC T2 Plus processors. That’s a total of 384 processing cores. If you do your math, that works out at 21,202 tpmC/core. In contrast, the most recent IBM result demonstrated 95,080 tpmC/core. In other words, using these two TPC-C results, you can conclude that for every 1 license of DB2 running on an over a year old POWER6 server, you’d need 4.5 cores of Oracle running on a latest and greatest Sun SPARC Server.
Furthermore, if you compare the “per core processing throughput” with Oracle’s best non-RAC TPC-C result, you will see that there is a significant reduction in throughput when using RAC. If you want, I can include details.
So, one possible conclusion is that RAC does not have a good scale-out story. And actually if you read through the detailed benchmark disclosure, you will find all sorts of interesting things. For instance you can see that Oracle actually turned off Cache Fusion for the benchmark. You will also see that Oracle attempted to improve perfomance by directing transactions to specific nodes using a mastering technique. Furthermore, you will see that Oracle turned off page integrity checking, again in order to improve performance. Would you run a production environment without page integrity checking?
Conor O'Mahony
November 2, 2009 at 11:01 am
Its now been 3+ months since Sun/Oracle released this world record TPC-C result.
So if DB-Purescale is so wonderful, if SSDs are now available @ IBM, Power7 is here, etc, why hasn’t IBM submitted a new earth shattering TPC-C result to beat Sun?oracles record?
Compared to IBM’s best TPC-C result with the Power 595 server and their fastest P6 processor, the Oracle-Sun TPC-C benchmark configuration and results demonstrated:
26 % faster performance
Oracle-Sun = 7,646,486 tpmC ¹; IBM = 6,085,166 tpmC ²
7x better transaction response time ³
Oracle-Sun = .168 seconds avg.; IBM = 1.22 seconds avg.
16 % better price/performance ¹
Oracle-Sun = $2.36/tpmC; IBM = $2.81/tpmC
8x fewer racks; less floor space
Oracle-Sun = 9 racks; IBM = 76 racks
10.7x better computational density 4
Oracle-Sun = 849,610 tpmC/rack; IBM = 80,067 tpmC/rack
4.6x less energy consumed
Oracle-Sun = 73 KW per hour; IBM = 343 KW per hour
5.9x better power/performance
Better availability with Oracle RAC and a cluster of 12 Sun T5440 servers; compared to a single IBM Power 595 server
What has IBM’s response been to the new Oracle-Sun world record TPC-C results?
- “Our per core tpmC performance is better.”
- “The Sun tpmC per core is 20,097; while the IBM tpmC per core is 95,080. The IBM result had 4.7 times higher performance.”
- “The Sun TPC-C configuration used 384 cores; while the IBM TPC-C configuration had 64 cores”.
- “Sun had to cluster 12 systems to beat the IBM single system result on TPC-C.”
Perhaps you have heard this argument from IBM before? Lets discuss some facts regarding these IBM claims and what it may mean to you in regards to your transactional database systems:
tpmC performance per core is not a metric of the TPC-C benchmark
tpmC performance per core does not predict better system transaction performance; nor better price/performance per system
As evident by the results of these TPC-C results from Oracle-Sun and IBM
What matters to a customer is the number of transactions per minute, cost and the reliability of those transactions, that can be delivered by their system architecture (servers, storage, software and networking)
Customers do not buy cores to run their transactional databases, they buy systems to deliver their required level of transactional performance and reliability.
Customers are also interested in the total price performance of the solution; and the Oracle-Sun result is 16% better than IBM in price per tpmC.
In regards to system transactional performance and price performance, does it matter if your system architecture used fewer faster cores or more slower core when the total price for the performance is less on one vendor’s solution versus the other’s?
If per core performance is a metric used by IBM to predict system performance, then IBM should also provide the publicly disclosed TPC-C TCO (hardware + software + 3 year maintenance) pricing on a per core basis as well. If they did, it would show that IBM is 5.6x more expensive per core in a configuration that delivers 26% less tpmC
Sun configuration price per core = $47,009/core
$18,083,745 configured price per TPC-C disclosure / 384 cores
IBM configuration price per core = $267,371/core
$17,111,768 configured price per TPC-C disclosure / 64 cores
Again, in regards to transactional performance, does it matter if your system architecture used fewer faster cores or more slower cores? or what the price per core is?
IBM will use per core performance to illustrate Oracle license cost differences between the Oracle-Sun and IBM configurations and results. IBM states these arguments:
Sun T5440 (12) = 192 Oracle licenses (384 cores / .5x Oracle license core multiplier)
Oracle Enterprise Edition, Oracle RAC, Oracle Partitioning, 3 years of maintenance
$19,601,280
IBM Power 595 = 64 Oracle licenses (64 cores / 1x Oracle license core multiplier)
Oracle Enterprise Edition, Oracle Partitioning, 3 years of maintenance
$4,701,120
IBM is misleading in their arguments per core performance and Oracle costs, relative to the TPC-C benchmark results:
tpmC price/performance/core is not a metric of the TPC-C benchmark
The published TPC-C disclosures already included the Sun and Oracle hardware, software and 3 year maintenance costs; and the Sun-Oracle price/performance per system was 17% better than IBM’s
IBM is making a flawed comparison of their Power 595 tpmC result using the DB2 database, by comparing the TPC-C configuration of Sun and IBM with the number of Oracle licenses
Why is IBM publishing Power 595 results with DB2, yet comparing Oracle licensing on the server?
IBM Power 595 has NO published benchmarks (not on any database related public benchmark) using the Oracle database to make this comparison with.
Phil
February 10, 2010 at 1:30 pm
[...] came from the admittedly biased Conor O’Mahony, a DB2 product manager for IBM. In his blog, Conor points out some interesting elements to Oracle’s pricing and support for the system [...]
Additional Caveats Obscure Oracle’s TPC Benchmark « Merv's Market Strategies for IT Suppliers
December 23, 2009 at 9:21 pm
Phil, I think what Conor is rightfully trying to point out (and it is of course a view I share…but I’m biased as well) is that when you look at TPC-C benchmarks, if you take the price and price/performance at face value, then you are doing yourself a grave injustice because of the pricing “gamesmanship” Oracle is using.
I don’t know of a single company that is paying only 1% of their license price for Oracle support…do you? If that were the case, the procurement person at that company would be the most sought after procurement officer in the world. But that’s the support price (which is factored into the total cost of the system and the price/performance metric). I’m not suggesting that it’s illegal, in fact it is right in line with the TPC-C rules, but I don’t think anyone is claiming that TPC-C rules are set up to reflect street pricing (otherwise you would see lots of customers paying 1% for support instead of 22%).
Now the argument for looking at just performance/core is that it takes some of the bravado out of the equation and gives you a metric that you can then apply to the costs you pay as a customer. I can easily calculate how much I’m paying per core for support and license costs for the products I own and then I can compare that to the raw performance per core and voila – I have a comparison in my own pricing terms (what I can negotiate and not some 3 year lease price which I don’t use).
Chris Eaton
February 10, 2010 at 4:31 pm
[...] is actually reported by the TPC Price/Performance metric (although you need to be aware of Sun and Oracle TPC Price/Performance Tactics, because IBM and Oracle use different tactics for TPC/C Price/Performance numbers). By focusing on [...]
Smarter Questions for a Smarter Planet » What Really Matters in a TPC-C Benchmark Result?
June 13, 2011 at 9:45 pm
[...] I also believe there is an error in their claim; the price/performance for the Oracle system is $4.20, the price/performance for the IBM system is $6.85 USD, which is actually 2/3 the price/performance and not the 1/3 Oracle are claiming. Their cost comparisons are even more troublesome when you consider: Sun and Oracle TPC Price/Performance Tactics Revealed. [...]
Don’t be Fooled by Oracle’s Smoke and Mirrors | Smarter Questions for a Smarter Planet
September 29, 2011 at 3:49 pm
meus parabéns
Maria José Menezes Soares Passabão
January 18, 2012 at 8:24 pm