Earlier this month, Oracle issued a press release regarding their latest results for the SAP® Business Intelligence-Data Mart (BI-D) Standard Application Benchmark. In the press release, Oracle claim that the new Oracle Database result surpasses the best IBM DB2 result by more than six times. While this statement is true, it is a little misleading. You see IBM does not perform the SAP BI-D benchmark tests for DB2 on the Linux, Unix, Windows, or z platforms. IBM performs these benchmarks only for DB2 on the i platform. So, when Oracle says that its result surpasses the best IBM DB2 result by more than six times, it is comparing a 4-node Oracle RAC cluster with a single DB2 for i system. Oracle is also comparing a recent benchmark result against an IBM result that is more than a year old. So, while these comparisons seem spectacular at first glance, once you look at the details you realize that they are not entirely fair.
Also, please note that the BI-D benchmark is a read-only benchmark. We have increasingly heard from clients that this benchmark is not realistic for their environments. IBM clients tell us that their systems typically have mixed workloads. For instance, demands for more current data typically result in trickle feeding, and they often need to rebuild cubes while queries are running. For these reasons, IBM is increasingly unlikely to perform the BI-D benchmark tests. Instead, clients are interested in the more realistic SAP BI-MXL which includes inserts, updates, and deletes with the queries.
The other thing to be aware of is that these benchmark results further illustrate the gap in levels of support for SAP applications provided by these database software products. With IBM DB2, you simply set one variable (DB2_WORKLOAD=SAP) and all the required settings are configured automatically for you. However, if you look at the benchmark submission package, you will see that these new benchmark results from Oracle use some interesting configuration settings. Undocumented and unsupported configuration parameters in Oracle Database begin with an underscore. Well, Oracle uses six of these undocumented and unsupported configuration parameters in these benchmark results. I’m sure its not reassuring for Oracle Database users to learn that Oracle needs to use undocumented and unsupported parameters to get optimal performance. The parameters in question are:
- _optimizer_cost_based_transformation= off
- _query_rewrite_fudge = 1
- _improved_row_length_enabled= FALSE
- _optim_peek_user_binds = FALSE
- _optimizer_autostats_job = FALSE
- _optimizer_save_stats = FALSE
There are some interesting settings in here. Apparently, they don’t trust their cost-based optimizer because they disable it. And they appear to be setting some sort of query rewrite fudge factor. I wonder what fudge factor you should choose for optimal performance in your environment. And, if they are disabling it, perhaps the row length setting does not improve things as its name suggests it might 
PS. Many thanks to Chris Eaton for his expertise and help with this blog post.
Pingback: Additional Caveats Obscure Oracle’s TPC Benchmark « Merv's Market Strategies for IT Suppliers
You should have relied on an Oracle DBA for interpreting these, instead of Chris Eaton, who appears to be an IBM Product Manager. Your interpretation of the meaning of the parameters is quite wrong. _optimizer_cost_based_transformation=off does NOT turn off the cost-based optimizer. They have NOT, I repeat, NOT, disabled it by setting this parameter.
And your “speculation” as to the other parameters is quite off-base. I’m sure you knew that when you wrote it, but you chose to make it sound better, right?
Typical sales rep(tile).
Hi sapdba,
The reference to “sales reptile” is amusing. I hadn’t heard that one before. However, I’m not sure what it refers to in this context. But thanks for including nonetheless.
Well, the larger point is that Oracle had to set undocumented arguments in the benchmark. Usually undocumented arguments are undocumented for a reason. For instance, Oracle officially advises against using undocumented arguments unless you are under the direction of Oracle Technical Support.
You can’t really blame us for getting those meanings wrong when they are undocumented and the names indicate otherwise
Regards,
Conor.
True, I can’t blame you for not knowing the meaning of the parameters, and I share your irritation at the fact that undocumented parameters exist at all. But when you don’t know the meaning of something, you don’t make assumptions as claims of fact, which you did. Unless, of course, the truth is not what you’re after. And you appear to be after simple claims that sound good, true or not. Just like a typical sales rep(tile).
And, as someone with direct experience in this area, I can say definitively that setting DB2_WORKLOAD=SAP is nice and convenient for marketing, but if there is any amount of real activity, the database will require much more tweaking and tuning than that to perform well.
Oh, I see what you are saying. I didn’t intend for the following paragraph to be read as a presentation of facts. I should have been more careful in the phrasing I used. Apologies if you, or anyone else, interpreted it this way. I intended for it to be a tongue-in-cheek commentary based on the names of those undocumented parameters. You’ve got to admit that, to people who do not use Oracle Database, some of those undocumented settings do look interesting / amusing based on the names alone…