Soloman Barghouthi
Eric Berg
Roman KharkovskiRecently I had a discussion with a certain retail company in US about their enterprise integration initiative. One of the things that struck me was the way they are comparing costs of potential ESB solutions. Here are some of the assumptions they are making when comparing the costs:
- “We asked both vendors for the quote of their ESB product for 24 Intel cores”
- “We asked two of our partners for the services quote to implement the solution using ESBs from IBM and Vendor Red”
- “We are assuming that license terms and conditions for the ESB software are the same between both vendors”
- “We assume that operational costs of IBM and Vendor Red ESB are more or less the same”
I have been in the middleware business for over 18 years now and unfortunately these kinds of assumptions are used over and over again. My discussion yesterday was about ESB, but I have seen these assumptions applied to databases, application servers, messaging, caching, portals and other kinds of enterprise middleware. How unfortunate. Not only this is not fare to vendors, but first and foremost these assumptions are counter productive for the very company that is evaluating the software. Let me consider these one by one and explain why you should NOT be using these in your decision process.
“We asked both vendors for the quote of their ESB product for 24 Intel cores”
What this really means is that the company has sent a request to vendors to quote a certain number of licenses for a fixed hardware configuration. What is wrong with this assumption? Aren’t all ESBs created equal? No, they are not.
If you shop for a car, would you ask for a quote on Ford Focus vs. Ferrari 458? These are both cars, but they have different characteristics and different speeds.
If you shop for a bike to race in triathlon, would you compare price on a bike from WalMart vs. a Felt or Cervelo bike? You can ride any of those bikes, but you sure can’t compare them on price alone.
Same thing with software. You just can not compare the license cost for a single core and assume you get the same performance out of that configuration using arbitrary ESB (or other middleware) product. Lets assume you need to handle 1 million transactions per hour. How many cores of the ESB product do you need to handle the workload?
IBM performance team publishes detailed performance reports for WebSphere MQ and WebSphere Message Broker (all found on Support Packs page). In my earlier article comparing WebSphere Message Broker and Oracle Service Bus, I have used an example of such report for WMB v8 on Linux x86. I have not seen any public performance data for Oracle Service Bus, nor tuning recommendations for specific platforms and protocols. The following table shows the message rates that were obtained for the different use cases when running on a IBM xSeries x3690 X5 with 2 x Deca-Core Intel Xeon E7-2860 2.27GHz processors:
Your company integration logic is likely to be a mix of workloads listed above. For the sake of discussion, let me assume that the average throughput of your application on WebSphere Message Broker is 9,000 transactions per second on the above configuration. This translates to about 450 transactions per second per processor core. 1M transactions per hour is roughly 278 transactions per second. This means you can handle your integration workload with a single core of Message Broker and still have capacity on that core to handle 40% more transactions.
Does this mean you would also need only a single core of the other vendor ESB product? Not necessarily. I have seen many performance POCs at many different customers in US and abroad where performance results of Message Broker were order of magnitude better than that of competition (in particular vendor Red ESB). I also did performance tests of my own where I have seen 2 to 5 times performance advantage of Message Broker over this other ESB product for workloads such as file parsing, database access (read and write), WebSphere MQ interaction and XML payload handling over HTTP and MQ protocols. Unfortunately I can’t publish my performance report since Oracle license agreement requires me to get their written permission to do so. And given the results, I know they wont approve…
You might say that ESB is a very special case and there is no industry standard for ESBs. What about Java EE servers? Those are all standards based and surely are not very different from each other. They all run on Java and all implement similar APIs. Why bother with the sizing? Wrong again. As you can see in this blog article that compares price performance of WebSphere vs. WebLogic – you do get different transactions per core ratio when running different vendor’s application servers. In the case of IBM WebSphere on Power7+ you get twice as many transactions per core compared to WebLogic on SPARC T5. To borrow from that article, here is a comparison of transactions per core for application servers (thank goodness SPEC.org is a public benchmark, so I do not need to get Oracle permission for this comparison):
Bottom line – you simply CAN NOT make an assumption that you need the same number of cores for different vendor products to handle the same workload.
“We asked two of our partners for the services quote to implement the solution using ESBs from IBM and Vendor Red”
Implementation costs are very difficult to estimate and comparing bids from different integrators is very hard. There are simply too many variables and risks involved. Never mind the risk of not ever finishing the project, such as this example of a failed multi-billion dollar Oracle implementation.
How do you compare the implementation costs then if you can’t fully trust your integrator bids? This is more art than it is a science, but at the very least, I would highly recommend that you try to implement at least a small pilot project yourself to get a feel for the software. After all it is your business and you need to have at least some first hand experience with the software. You do not need to do a full implementation yourself, it is perfectly fine to hire 3rd party integrator to do the bulk of the work for you, but you are the one ultimately responsible for the implementation.
Download the software, install it and try to build something with it. Spend no less than one week with one vendor product and one week with another – just to get a feel for those products. You might be surprised. Personally I find WebSphere Message Broker very easy to use. Sure, there is a learning curve, but once you get a feel for the product, it becomes very productive. I recorded several demos with Message Broker and Oracle Service Bus – implementing the same integration logic in both products. What takes me 6 minutes and 50 mouse clicks in Message broker, takes over 1 hour and 1,000+ mouse clicks in Oracle Service Bus. Unfortunately I can’t post my demos in the public domain because Oracle license agreement is very restrictive. But you can get a feel for the Oracle Service Bus productivity from this article (not sponsored, nor affiliated with IBM in any way).
“We are assuming that license terms and conditions for the ESB software are the same between both vendors”
I have covered this topic in my other posts and will simply refer you to these articles:
- http://smarterquestions.org/2012/02/how-to-cut-your-weblogic-license-and-maintenance-costs-in-half
- http://smarterquestions.org/2012/02/ibm-and-oracle-software-licensing-and-support-in-virtualized-private-cloud-environments
“We assume that operational costs of IBM and Vendor Red ESB are more or less the same”
Operational costs include installation, monitoring, deployment, configuration and many other factors. I will cover these in my future posts in more details, but for now let me make few quick notes on this subject:
- Installation time of Message Broker is only about 3 minutes as I have documented in this demonstration video. Try installing Oracle Service Bus with all of its prerequisites and compare that to Message Broker. (Tip: You may be able to get around some of the complexity of Oracle install by using shared files system with binaries.)
- Another consideration is the number of cores and number of machines machines you need to manage. Performance differences require configurations of different sizes and the number of machines in a cluster has direct impact on the difficulty of administration.
- Performance tuning of Message Broker is well documented in IBM performance reports and public best practices documents. Tuning Oracle Service Bus involves tuning of the JVM, WebLogic Server and Oracle Service Bus itself. Outside of the basic Oracle documentation, there is not much information available to the public on best practices and performance of Oracle Service Bus, nor can you find a single performance or sizing guide report from Oracle on the Oracle Service Bus.
Conclusion
There are many considerations that need to be evaluated when selecting your integration bus and the cost is very important, however one needs to be thoughtful on how to compare costs between different vendors and products. In any case it is a lot more than just comparing a cost for a single core license. You may want to consider experience of companies who decided to move from Oracle to IBM middleware to reduce their costs.
