Archive for the ‘Advertising’ Category
Last week, I was in Dallas speaking at an event. In the morning, as I left my hotel room, I picked up the Wall Street Journal which was outside my door. I was surprised to see that Oracle are re-running an old advertisement:
Why was I surprised? Because this advertisement is based on an industry benchmark that shows that Oracle uses 9 times the number of CPU cores to achieve only 3 times the performance of the IBM result. To put it another way, if you look at the per-core throughput, the IBM system is 3 times faster than the Oracle system. Oracle highlight the overall throughput of the system, but if you do some investigating you will see that the Oracle system in question uses 1,728 CPU cores, whereas the IBM system in question uses only 192 CPU cores. Considering that you typically pay for software based upon the number of CPU cores, I know which system I’d prefer to be buying software for
By the way, if you want to see how big these benchmark systems are, check out this blog post… TPC-C Result in Real World Terms: Big Macs and Walmart. Of course, while the benchmark systems themselves are—for the most part—disconnected from today’s real world situations, that is not to say that they are not useful. They are. They serve a very useful purpose in stress testing the different vendor’s products. And they also demonstrate how efficiently those systems scale out.
This is why I’m surprised that Oracle is persisting with advertising this benchmark result. For fun, let’s create a graph that doesn’t show the overall throughput of the systems. Let’s instead create a graph that shows the throughput per CPU core for these benchmark systems. Some people might consider this to be a good measure of efficiency for the systems. As you can see, when you look at this measure of efficiency, it paints a very different picture (of course, the higher the number, the better).
Results on Transaction Processing Performance Council Web site at http://www.tpc.org. Results as of 06/08/11.
Oracle SPARC SuperCluster (108 chips, 1728 cores, 13824 threads); 30,249,688 tpmC; $1.01/tpmC; available 6/1/11.
IBM Power 780 cluster (24 chips, 192 cores, 768 threads); 10,366,254 tpmC; $1.38/tpmC; available 10/13/10.
HP Integrity Superdome (64 chips, 128 cores, 256 threads); 4,092,799 tpmC; $2.93/tpmC; available 08/06/07.
The second of the series of DB2 ads that compares IBM DB2 with Oracle Database has been released. Here it is…
You can find out more about these ads, and the details behind the claims at ibm.com/facts.
Today, IBM is launching a new IBM DB2 vs. Oracle Database advertising campaign. This camapign will run in both print and online media. Much of the content in the current and upcoming ads will be familiar to readers of this blog. I hope you enjoy them…
Remember the infamous advertisement from Oracle where they claimed they would be announcing a TPC-C benchmark result with XX million tpmC, hinting at a massive double digit TPC-C result. And then they ended up announcing a result with more than 7 million tpmC. At the time, a few people joked that they didn’t expect the first X in Oracle’s boast to be a zero. Well, last week IBM became the first vendor to break the 10 million tpmC barrier for the TPC-C benchmark.
The new benchmark result is impressive in many regards. It provides 35% greater throughput1 than the Oracle/Sun result. But this great leap forward in throughput is only part of the story. An equally important consideration is system efficiency, with the IBM result having only half the number of CPU cores when compared with the Oracle/Sun system. If you do the math, the IBM result gets 2.7 times more productivity per CPU core than the Oracle/Sun result. And because software is typically licensed by CPU core, that has the potential to add up to a lot of savings, both for initial purchase price and for ongoing maintenance costs.
But the aspect of this result that really astounded me was the price/performance metric. The IBM result achieved 41% lower cost per transaction than the best Oracle/Sun TPC-C performance result. And don’t forget the games that Oracle played in their price/performance metric. This means that IBM is 41% better even when Oracle resorts to these tactics!
You may recall that, when Oracle talked about their benchmark result, they boasted that their system needed less energy than IBM’s system. Of course, what they neglected to mention was that their system used Solid State Disk (SSD) technology and, at the time, IBM’s used only spinning disk. Well, now the IBM system also uses SSD technology. With this level playing field, the IBM system requires 35% less energy per transaction (Watts/tpmC) than published Oracle energy usage data2.
I’m sure that, like me, everyone in the DB2 community is delighted that DB2 has reclaimed its position at the top of the TPC-C benchmark in such an emphatic manner. It is further proof of DB2′s performance leadership. In fact, since 1 January 2003, DB2 has enjoyed more time leading the TPC-C, TPC-H 10TB, and SAP 3-Tier SD benchmarks than all other vendors combined.
2 Energy claims for either system are not related to official TPC-Energy results and should not be compared to TPC-Energy results. Energy comparisons are between IBM and Oracle/Sun system configurations referenced above. IBM POWER7 energy consumption = 65130 Watts, 0.006282 Watts/tpmC; Oracle/Sun system consumption = 73932 Watts, 0.009668 Watts/tpmC. IBM energy estimate based in IBM calculations using customer-available energy estimation tools for IBM servers, storage energy estimation reports available from IBM Techline services, and published component active power consumption specifications. Oracle energy estimate from Oracle-published results available at http://www.oracle.com/features/strategic-focus-report.pdf.
Many DB2 users I talk to have seen provocative print ads from our competitors and asked why IBM don’t create ads like those. For various reasons, IBM won’t ever create those kind of ads. They simply don’t gel with our mission. Isolating and exploiting individual speeds-and-feeds, and doing so out of context, is not what IBM is about.
If you are interested in learning what we are about, you need look no further than our senior leadership. Listen to them talk. Listen to the IBM earnings calls. It’s all about client value. And this focus on providing client value starts at the top of our organization and pervades throughout our business. What do I mean by client value? Well, it’s about helping our clients accomplish their goals, and providing them with good value for their IT investment. One of the best ways to be successful is to enable your clients to be successful. By helping our clients to achieve or exceed their goals, and by providing enduring value, IBM has created a business that has seen many competitors come-and-go over the years.
You can see this approach reflected in our new advertising campaign. I include an image showing the two-page ads below. Yesterday, these appeared in the Wall Street Journal, New York Times, San Francisco Chronicle and San Jose Mercury News. They will appear in magazines like Business Week, Forbes, Fortune and Information Week in the weeks to come. Here is the main paragraph of the ad…
What exactly does a benchmark mean? For the last five years, IBM DB2® has spent more time
at the top of three of the industry’s leading performance benchmarks than Oracle and Microsoft
combined*. But is that the best way to think about the possibilities of technology? What really
matters isn’t some abstract measure of performance, it’s what companies actually do with that
performance. For instance, Globe Telecom is using a service delivery platform from IBM to
increase their sales by 112%. EuResist is using an integrated analytics solution to predict the
most effective drug combinations for individuals with HIV, with 78% accuracy. And CAIXA
Econômica Federal, one of the largest banks in Latin America, is using a service oriented
architecture to slash infrastructure acquisition costs by over $330 million. On a smarter planet,
these are the benchmarks that matter.
Last year, I blogged about some issues regarding Truth in Advertising – Advanced Data Compression in Oracle 11g. Well, our friends over at Oracle are at it again. This time thay are making some questionable advertising claims. Here is the ad in question. From looking at the ad, you would think that an Oracle/Sun system gives you seven times the performance, while consuming one sixth the amount of power. Let me explain why this is MISLEADING.
There are two claims here, both under the banner of being “independently verified.” The independent verification refers to the fact that it is drawing from TPC benchmark data.
The first claim pertains to performance. When someone mentions performance to me in relation to the TPC-C benchmark, I immediately think of the primary metrics, and in tpmC in particular. After all, this is a primary metric for a reason. tpmC represents the performance of the systems for all workloads (for the record, Oracle did outperform IBM by about 20% for tpmC). But, Oracle obviously aren’t looking at tpmC when devising this claim. Instead, they are focusing only on one subset of the performance numbers in the benchmark. In other words, if the TPC-C benchmark were like a triathalon, then Oracle did really well in one of the events. It is downright misleading for them to claim that a 7x lead in one event is indicative of their performance in the overall race.
By the way, you should also be aware that Oracle are comparing an older IBM system to their latest and greatest, which is questionable in its own right. With the rate of change in the industry, IBM’s 2008 result is not indicative of its performance levels today. In addition, the Oracle configuration actually uses 115 TB of solid-state disk (for a database size of 6TB). The IBM result does not use any solid-state disk, instead working with mechanical disks. Solid state disk manufacturers claim that their products are hundreds of times faster than mechanical disk. However, for Oracle, that translated into only a 20% lead over IBM.
But, believe it or not, this is not as misleading as the second claim pertaining to energy consumption. First of all, the TPC results being touted here were posted before the TPC-Energy metric was introduced and reported. This energy data is not coming from the TPC results. Putting this claim under the “independently verified” banner is simply misleading.
Let’s dig a little deeper and do some math with the server specs. Note that Oracle needed a cluster of 12 SPARC Enterprise T5440 servers for their benchmark result, whereas IBM needed only one IBM Power 595 server.
If you go to the Sun SPARC T5440 Power Calculator, you can see that a single server consumes between 1551 watts (idle) and 2002 watts (100% active). There are 12 of these servers in Oracle’s benchmark, which results between 18.612 KW and 24.024 KW of power consumption.
If you look at the same information for the IBM POWER 595, you will see that during typical usage a P595 consumes 18.5kW. At 100% utilization, it consumes 27.7kW.
That’s right, the Oracle configuration in an idle state consumes more power than the IBM configuration performing a typical workload. Oracle, please explain how you arrived at the 6x number in the ad…
Ryan Tate claims that “Larry Ellison will always be a shameless truth-bender” in an article titled Larry Ellison Can’t Be Bothered With the Facts. The article talks about an advertisement that appeared recently in the Wall Street Journal and goes on to share some interesting anecdotes about Oracle advertising.
The advertisement in question suggests that Oracle plan to announce a new TPC-C result using Sun hardware on October 14. One thing this article does not mention is that, as far as I can tell, the advertisement does not follow the TPC rules. If you read Section 8.2.2 Unfair Use, you will see that Oracle could be interpreted to have violated several of these rules and unfairly used the TPC benchmark. You see, the TPC does not allow organizations to “make TPC-related claims or lead the reader to TPC-related conclusions which are untrue or cannot be substantiated by the entire body of results.” TPC does this to prevent misleading advertising. You may wonder how this could be misleading. Well, it could be misleading because it does not tell the full story. For instance, what if their upcoming benchmark result requires an inordinately expensive hardware configuration. Without certain primary metrics, TPC results cannot reasonably be compared.
Aside from Oracle’s apparent flaunting of the TPC rules, this advertisement is also notable because Oracle indicates that the upcoming TPC result will be on Sun hardware. The reason that this is interesting is that Sun has publicly derided the TPC-C benchmark in the past. In fact, they have not participated in the TPC-C benchmarks for several years, claiming that the benchmark does not reflect real world transactional workloads, and that the benchmark is easily gamed by vendors. It is an interesting about-face for Sun. I wonder what prompted this change in what was a very firm stance on their part.
You may have seen the following advertisement from Oracle. They claim that Advanced Data Compression for Oracle 11g cuts your data storage requirements in half. Anyone who knows anything about data compression know that this is misleading. Sure, its possible that data compression could cut your data storage requirements in half. But that depends on the kind of data you have and on your environment. As we will see below, compression rates vary greatly depending on the nature of the data.
Let’s take a quick look at the results of an InfoWorld investigation at Oracle Database 11g Advanced Compression Testbed, Methodology, and Results. In this investigation, InfoWorld analyzes two data sets: 1) the test kit that Oracle provides and 2) several tables from the TPC-C order entry benchmark data.
Let’s focus on the industry benchmark data. The first thing to notice is the variation in the compression rates. They vary greatly, depending on the nature of the data. The second thing to notice is that if do your math and figure out the before and after storage numbers, you will see that Advanced Data Compression for Oracle 11g reduces the total storage for these tables from 273,034 MB to 210,808 MB. This is a long way short of the 50% number that Oracle is touting! In fact, its actually saving less than 23%. However, as I have said, compression rates do vary greatly depending on the nature of the data and TPC-C data does contain random strings which do not compress well.
|Table Name||Original Size (MB)||Compression Rate (%)|
The next thing to notice in the InfoWorld article are the performance findings. While Oracle claims that “it runs faster” in the advertisement, the results of the InfoWorld investigation did not back this up. In fact, the InfoWorld investigation observed transactions per second for a mixed workload “falling from 28 to 13, which “represents a nearly 50% performance hit“. However, InfoWorld do point out that their tests are not a true representation of a real-world workload, and that the results are within acceptable parameters.
The point of this blog post is not to question whether Oracle can achieve high compression rates or whether performance is acceptable when compression is turned on. The point of this blog post is to make sure you are aware that compression rates and performance are highly dependent on the nature of your data and the nature of your environment, and don’t let vendors like Oracle tell you otherwise.
IBM recently created some nice new horizontal and vertical Web banners for DB2. You can see the new banners on the sidebar and footer of the Information Agenda Website. If you refresh that Web page, you will likely see additional new banners. Here is my favorite of those banners: