Conor O'Mahony's Database Diary

NYSE Euronext operates multiple securities exchanges, including the New York Stock Exchange and Euronext. As you might imagine, securities exchanges present significant data management challenges. But NYSE Euronext didn’t just want to have a transactional system, they wanted to do much more with their data, further increasing the challenges. At the 2011 IBM Information On Demand (IOD) conference, NYSE Euronext described their challenges and the solution they chose. In particular, they highlight Netezza’s tremendous performance and how fast it is to get up-and-running with Netezza.

Not only is it easy to get up-and-running with Netezza, but it is easy to manage your environment on an ongoing basis. You can hear for yourself in this short video segment…

Yesterday, Oracle announced a new TPC-C benchmark result. They claim:

In this benchmark, the Sun Fire X4800 M2 server equipped with eight Intel® Xeon® E7-8870 processors and 4TB of Samsung’s Green DDR3 memory, is nearly 3x faster than the best published eight-processor result posted by an IBM p570 server equipped with eight Power 6 processors and running DB2. Moreover, Oracle Database 11g running on the Sun Fire X4800 M2 server is nearly 60 percent faster than the best DB2 result running on IBM’s x86 server.

Let’s have a closer look at this claim, starting with the first part: “nearly 3x faster than the best published eight-processor result posted by an IBM p570 server“. Interestingly, Oracle do not lead by comparing their new leading x86 result with IBM’s leading x86 result. Instead they choose to compare their new result to an IBM result from 2007, exploiting the fact that even though this IBM result was on a different platform, it uses the same number of processors. Of course, we all know that the advances in hardware, storage, networking, and software technology over half a decade are simply too great to form any basis for reasonable comparison. Thankfully, most people will see straight through this shallow attempt by Oracle to make themselves look better than they are. I cannot imagine any reasonable person claiming that Oracle’s x86 solutions offer 3x the performance of IBM’s Power Systems solutions, when comparing today’s technology. I’m sure most people will agree that this first comparison is simply meaningless.

Okay, now let’s look at the second claim: “nearly 60 percent faster than the best DB2 result running on IBM’s x86 server“. Oracle now compare their new leading x86 result with IBM’s leading x86 result. However, if you look at the benchmark details, you will see that IBM’s result uses half the number of CPU processors, CPU cores, and CPU threads. If you look at performance per core, the Oracle result achieves 60,046 tpmC per CPU core, while the IBM result achieves 75,367 tpmC per core. While Oracle claims to be 60% faster, if you take into account relevant system size and determine the performance per core, IBM is actually 25% faster than Oracle.

Finally, let’s not forget the price/performance metric from these benchmark results. This new Oracle result achieved US$.98/tpmC, whereas the leading IBM x86 result achieved US$.59/tpmC. That’s correct, when you determine the cost of processing each transaction for these two benchmark results IBM is 39% less expensive than Oracle. (BTW, I haven’t had a chance yet to determine if Oracle Used their Usual TPC Price/Performance Tactics for this benchmark result, as the result details are not yet available to me; but if they have, the IBM system will prove to be even less expensive again than the Oracle system.)

Benchmark results are as of January 17, 2012: Source: Transaction Processing Performance Council (TPC), www.tpc.org.
Oracle result: Oracle Sun Fire X4800 M2 server (8 chips/80 cores/160 threads) – 4,803,718 tpmC, US$.98/tpmC, available 06/26/12.
IBM results: IBM System p 570 server (8 chips/16 cores/32 threads) -1,616,162 tpmC, US$3.54 /tpmC, available 11/21/2007. IBM System x3850 X5 (4 chips/40 cores/80 threads) – 3,014,684 tpmC, US$.59/tpmC, available 09/22/11.

During this week’s keynote address at the International DB2 User Group (IDUG) conference in Prague, Namik Hrle talked about Storage Class Memory. Storage Class Memory is a technology in development that promises the performance of Solid State Drive (SSD) technology at the low cost of Hard Disk Drive (HDD) technology. It also promises compelling breakthroughs in space and power consumption. Storage Class Memory is essentially the marriage of scalable non-volatile memory technology and ultra high-density technology. Here is a table that projects the 2020 characteristics of Storage Class Memory:

This table was actually created in 2008. From what Mr. Hrle says, we are tracking ahead of this schedule and will have these capabilities available sooner than 2020.

The performance limitations of disk-based systems have led to the addition of many database and data warehouse “features” (clever optimizations that address these limitations, and provide acceptable performance). If Storage Class Memory delivers on its random and sequential I/O performance promises, as well as its cost promises, many of these optimizations will become either less important, or perhaps unnecessary. In fact, it makes you wonder if our industry’s current fixation with in-memory capabilities may be short-sighted. Several vendors have in-memory database product visions that will not be realized until the latter half of this decade, which is a similar time frame to the projected availability of low-cost Storage Class Memory. Certainly food for thought…

The American Association of Retired Persons (AARP) recently paired Netezza with their transactional environment, which includes DB2 for z/OS, and achieved remarkable results. Often when you read customer success stories, you are bombarded with metrics. The AARP success story has those metrics:

• 1400% improvement in data loading time
• 1700% improvement in report response time
• 347% ROI in three years

But metrics tell only part of the story. And sometimes the story gets a lot more interesting when you dig a little deeper.

AARP had been using Oracle Database for their data warehouse. But their system simply could not keep up with the demand. As Margherita Bruni from AARP says, “our analysts would run a report, then go for coffee or for lunch, and, maybe if they were lucky, by 5:00 p.m. they would get the response—it was unacceptable—the system was so busy writing the new daily data that it didn’t give any importance to the read operations performed by users.” The stresses on their system were so great that in 2009 alone, their Oracle Database environment had more than 30 system failures. To compound matters, these system performance issues meant that full backups were not possible. Instead AARP would back up only a few critical tables, which is a less than desirable disaster recovery scenario. Clearly, something had to be done.

AARP chose to move their 36 TB data warehouse to Netezza. You can see from the metrics above that they achieved remarkable performance improvements. But what do those performance improvements mean. Well, for the IT staff, they mean that they are relieved of a huge daily burden. Their old system required one full-time database administrator (DBA) and one half-time SAN network support person. These people are now, for the most part, free to work on other projects. And more importantly, they don’t have to deal with the stress of the old environment any more.

But the benefits are not being enjoyed only by the IT staff. They are also being enjoyed by the business analysts, who according to Bruni “could not believe how quickly results were provided—they were so shocked that their work could be accomplished in a matter of hours rather than weeks that, initially, they thought data was cached.” She goes on to say that “one analyst, who is now a director, told us that he used the extra time for other projects, which ultimately helped him become more successful and receive a promotion.” Now that is what I call a great impact statement. The metrics are great, but when someone is freed up to do work that gets them a promotion, that’s a very tangible illustration of the difference that Netezza can make.

Another illustration of the difference is the impact it had on the group that implemented the Netezza system. As Bruni says “after we moved to IBM Netezza, the word spread that we were doing things right and that leveraging us as an internal service was really smart; we’ve gained new mission-critical areas, such as the social-impact area which supports our Drive to End Hunger and Create the Good campaigns.” It certainly looks like you can add IT management to the list of constituents who have had a positive career impact as a result of moving from Oracle Database to IBM Netezza.

For more information about this story, see AARP: Achieving a 347 percent ROI in three years from BI modernization effort.

AMT-SYBEX are a leading provider of platforms for traditional and smart metering. They created a Meterflow Benchmark to help customers choose the best underpinning infrastructure for their platform, and they worked with IBM to run that benchmark with Informix TimeSeries. I previous blogged about Why Informix Rules for Time Series Data Management. Well, the results of this benchmark further illustrate the benefits of Informix TimeSeries. The following quote is from the resulting AMT-SYBEX case study:

We believe that this represents ground breaking levels of performance which is ten times faster than other published benchmarks in this area.

As you can see, Informix is 10x faster than the leading database software they previously worked with. If you read the Executive Summary, you will also see that IBM Informix enjoys almost linear scalability when going from 10 million meters up to 100 million meters, which is a great testament to the efficiency of operation for Informix TimeSeries.

A couple of years ago, IBM introduced the pureScale feature, which provides application cluster transparency (allowing you to create shared-disk database clusters). At the time, IBM had taken their industry-leading clustering architecture from the mainframe, and brought it to Unix environments. IBM subsequently also brought it to Linux environments.

Today, IBM announced its first public industry benchmark result for this cluster technology. IBM achieved a record result for the SAP Transaction Banking (TRBK) Benchmark, processing more than 56 million posting transactions per hour and more than 22 million balanced accounts per hour. The results were achieved using IBM DB2® 9.7 on SUSE Linux® Enterprise Server. The cluster contained five IBM System x 3690 X5 database servers, and used the IBM System Storage® DS8800 disk system. The servers were configured to take over workload in case of a single system failure, thereby supporting high application availability. For more details, see the official certification from SAP.

Column-oriented database systems (Column Stores) have attracted a lot of attention in the past few years. Vendors have quoted impressive performance and storage gains over row-oriented database systems. In some cases, vendors have even claimed as much as 1000x performance improvement.

For many years Sybase IQ led the way for Column Stores. But they have since been joined by a long list of vendors, including Infobright, Paraccel, and Vertica. The claims being made by these vendors are attention-grabbing. But they don’t tell the whole story. Before you run out and get yourself a Column Store, you should be aware of the following:

• Performance issues when queries involve several columns.
  Column stores can be faster than row-oriented stores when the number of columns involved is small (it depends on a number of factors, including the number of columns, the use of indexing, the specifics of the query, and so on). However, you can run into performance issues with Column Stores as the number of columns increases, due to the re-composition overhead. In fact, the performance degradation can be quite significant. If your queries involve more than a few columns (either columns for retrieving data or columns for query predicates), you need to be aware of this potential issue. If you are evaluating a column Store, make sure to test queries that involve more than a couple of columns.
• Overhead associated with inserting data.
  When you create a new data record, you are creating a data row. However, a Column Store does not have a row-orientation. Instead, a Column Store must decompose that data row into the individual column values, and store each of those column values individually. This adds up to a lot more block updates for a Column Store than a row-oriented store. As you can imagine, this is quite a bit of additional work. You should also keep in mind that the values in Column Stores are typically sorted for fast selection and retrieval, which means even more work for data insert and update operations. So what does all this mean? Essentially these limitations make it difficult to have real-time or near real-time data analysis with Column Stores (unless the Column Store vendor uses an approach like Vertica where they have an “update area” in memory that is essentially a row store cache, where real-time inserts are first written, then asynchronously written to disk).

Column Stores are great as analytic data marts where queries do not involve many columns. In such situations, you can enjoy performance gains. However, for more involved usage, you may run into issues. For instance, a Column Store is almost certainly not up to supporting thousands of simultaneous users and mixed query workloads, which are common in Enterprise Data Warehouse (EDW) environments. Sometimes people can get blinded by Column Store success for relatively simple data mart environments. You should be aware that these performance gains do not necessarily translate to larger, more complex environments. In fact, they may not even translate to other simple data marts with different schemas, or where your queries involve more than a couple of columns. The bottom line here is that you need to know both the benefits and the limitations of a Column Store, and make the right decision for your particular situation.

This is a presentation I put together a while ago, but for the most part the information still applies, and I thought some of you may find it interesting or relevant.

Here is a video where Philip Howard, Research Director at Bloor Research, evaluates performance, scalability, administration, and cost considerations for IBM Smart Analytics System and Oracle Exadata [for data warehouse environments]. This video is packed with great practical advice for evaluating these products.

Philip Howard, Research Director at Bloor Research, recently evaluated the performance, scalability, administration, and cost considerations for the leading integrated systems from IBM and Oracle for OnLine Transaction Processing (OLTP) environments. Here is a summary of his conclusions:

And here is a video with his evaluation. It is packed with practical advice regarding storage capacity, processing capacity, and more.

Last year, I blogged about how IBM DB2 has the Leading x86-based TPC-C Result. Well, IBM has further cemented DB2′s position as the leading database for x86-64 systems with a new record-breaking TPC-C benchmark result. The new benchmark result achieved more than 3 million transactions per minute on an IBM System x 3850 X5. The entire system for this result is housed in a single, space-saving 42U rack. The system runs DB2 9.7 on SUSE Linux. It has four Intel Xeon E7-8870 processors running at 2.40GHz (4 processors/40 cores/80 threads). It should also be noted that the system uses Solid State Drive (SSD) storage for faster database access.

For me, the most interesting aspect of this result is not just the performance; it is the price for that performance. And, of course, price/performance is a key consideration for all systems, but especially for cost-conscious x86-64 purchasing decisions. The new system costs US$0.59 per tpmC. As of today, this is the lowest cost of any system in the Top 10 TPC-C performance results (by the way, the next lowest cost also features DB2). See for yourself at TPC-C – Top Ten Performance Results.

If you look at the TPC-C – Top Ten Price/Performance Results, you will see some results from Oracle that offer better price/performance. However, these Oracle results are for very small benchmark systems; approximately one-tenth the size of the IBM DB2 systems. And they include only Oracle Database 11g Standard Edition One; whereas the IBM results include the full enterprise edition of DB2. Not only do the IBM benchmark system give you more product capability for your money, but you can clearly see that the performance of the IBM systems and the cost per transaction for the IBM systems both scale up very nicely.

IBM System x®3850 X5 (Intel Xeon E7-8870 processors 2.40GHz, 4 processors/40 cores/80 threads) TPC-C result of 3,014,684 tpmC, $.59 USD/tpmC, available 9/22/11, DB2 9.7, SUSE Linux® Enterprise Server 11 (SP1)

Last week, I was in Dallas speaking at an event. In the morning, as I left my hotel room, I picked up the Wall Street Journal which was outside my door. I was surprised to see that Oracle are re-running an old advertisement:

Why was I surprised? Because this advertisement is based on an industry benchmark that shows that Oracle uses 9 times the number of CPU cores to achieve only 3 times the performance of the IBM result. To put it another way, if you look at the per-core throughput, the IBM system is 3 times faster than the Oracle system. Oracle highlight the overall throughput of the system, but if you do some investigating you will see that the Oracle system in question uses 1,728 CPU cores, whereas the IBM system in question uses only 192 CPU cores. Considering that you typically pay for software based upon the number of CPU cores, I know which system I’d prefer to be buying software for

By the way, if you want to see how big these benchmark systems are, check out this blog post… TPC-C Result in Real World Terms: Big Macs and Walmart. Of course, while the benchmark systems themselves are—for the most part—disconnected from today’s real world situations, that is not to say that they are not useful. They are. They serve a very useful purpose in stress testing the different vendor’s products. And they also demonstrate how efficiently those systems scale out.

This is why I’m surprised that Oracle is persisting with advertising this benchmark result. For fun, let’s create a graph that doesn’t show the overall throughput of the systems. Let’s instead create a graph that shows the throughput per CPU core for these benchmark systems. Some people might consider this to be a good measure of efficiency for the systems. As you can see, when you look at this measure of efficiency, it paints a very different picture (of course, the higher the number, the better).

Results on Transaction Processing Performance Council Web site at www.tpc.org. Results as of 06/08/11.
Oracle SPARC SuperCluster (108 chips, 1728 cores, 13824 threads); 30,249,688 tpmC; $1.01/tpmC; available 6/1/11.
IBM Power 780 cluster (24 chips, 192 cores, 768 threads); 10,366,254 tpmC; $1.38/tpmC; available 10/13/10.
HP Integrity Superdome (64 chips, 128 cores, 256 threads); 4,092,799 tpmC; $2.93/tpmC; available 08/06/07.

The second of the series of DB2 ads that compares IBM DB2 with Oracle Database has been released. Here it is…

You can find out more about these ads, and the details behind the claims at ibm.com/facts.

This is the conclusion of my series of blog posts about the Solitaire Interglobal research, which measures various aspects of database environments. In this post, I’m going to focus on performance and cost in IBM Power Systems environments.

Solitaire examined database performance in 1,430 production environments that use IBM Power Systems. You can see the specific breakdown on the counts of the different types of systems in the full report. Their research includes production systems for credit card processing systems, CRM systems, transaction processing systems, and DSS systems.

Here are the summary performance findings for the credit card, CRM, and transaction-processing systems. They indicate the average number of Transactions Per Second (TPS) for these systems. As you can see, DB2 appears to offer a clear performance advantage over Oracle Database. The full report includes details of the number of TPS for each production system in the analysis.

And here are the summary performance findings for the Decision Support System (DSS) environments, which use an Average Queries per Minute metric.

Solitaire also determined the operational costs for these environments. These are the costs for infrastructure and staffing. It does not include overhead costs like facilities, acquisition, and initial deployment. As you can see, the operational costs for IBM DB2 compare very favorably with Oracle Database, especially when you consider the superior performance of the DB2-based systems.

And when you include overhead costs to determine the overall costs, as you might imagine, DB2 offers even better value.

You can read and download the full Solitaire report at Comparing Real World Database Performance: IBM® DB2® versus Oracle® Database and Microsoft SQL Server®.

It is difficult to get direct performance comparisons between Oracle Exadata and competing products. Last month, The Register published what may be such a comparison in its IBM: Our appliance servers smoke Ellison’s ‘phony baloney’ article. They include an image that compares the performance and price of the IBM Smart Analytics System with a leading competitor. The IBM Smart Analytics System is, of course, an integrated hardware/software system for data warehousing and analytics that is based on DB2. The article covers an IBM Investor Day presentation that was delivered by Steve Mills of IBM, and includes the following explanatory passage:

“We benchmark all the time,” Mills said, and he pulled out some real tests to support his point. “We have a favorite competitor who likes the color red. We like the color blue. This is real workload benchmarking, not some phony baloney made-up thing that goes in an ad. We deliver a system that is fast for what customers run.”

Here is the chart that is included in the article. The Register assert that the competitor in red is Oracle Exadata.