Realtime Analytics for the rest of us

My two cents added to the discussions here and here.

John's points about the DW purchase decision being pushed to the SaaS vendor and being less relevant to the enterprise (analytic application mid-market customer) is the key to the big switch. A similar analogy would be that when an Enterprise gets its electricity from the electrical company (the GRID ). All that it cares about is the SLA – Can the supplier meet my XXX Megawatt per hour demands at peak loads of YY, 24/7 and after that the pricing is the decision point. What kind of generators the electrical company uses (Hydro , coal , windmills) is an important decision for the electrical company but certainly not relevant to the Enterprise using the electricity.

So this analogy leads to a bit of soul-searching for DW appliance vendors like Terradata , Greenplum etc. “Who is their customer?” are they taking up the role of GE (who manufactures turbines , windmills etc) to serve the SaaS vendors OR do they want to be offering solutions at a level higher to the end consumer that ultimately end up using their appliances. I think a little bit of both and mixture of a lot more partnerships is probably what is going to happen.

It is also interesting to note two other trends that will shape the BI world.

1. Fragmenting of DB market to specialized Database
   
2. Availability of webscale level specialized Databases like BigTable , Hadoop/Hbase at very low entry points.

These trends will lead to a development of a longtail-type market for real-time analytics in an SaaS model (example – Recommendation service based on collaborative filtering, using Predictive modeling results during the underwriting workflow for approving a quote).The reason this will happen is because

a) These kinds of applications are more focused and can be performed in silos’. The whole concept widgets moving at the next layer of functionality and reuse.

b) They are also better served by vendors whose livelyhood depends on bettering the algorithms that power the analytics engines.

c) With things like Bigtable , Hadoop/Hbase exposed to the world at a very low entry point , all it takes is one guy to improve an algorithm and expose to the world as a service.

Update: May 22 ,2008 : I ran into one more Cloud computing Data analytics solutions which is another Column oriented database check it out at Vertica

Why did SAP buy Business Objects?

Recently I ran into a client who had made a significant investment in Business Intelligence infrastructure and a decision support system. This Insurance Carrier had 8000 reports in their portfolio. Yes once again the number is 8000 ?? Needless to say that one of reason I was engaged was to rationalize the reports(besides fixing other problems they had like Data quality)


I work in the Information Management space and have come to realize that 80% of our work is in getting the infrastructure for decision support right (i.e focussing on ETL , Data Quality , one version of truth etc). A successful project's end point is generally the ability for the Business users to get the information they are looking for accurately.This measure of success traditionally has focussed most of the BI players and consultants to be technology centric and tools heavy on the plumbing aspect for data.


So where does SAP(NASDAQ:SAP) and Business Object's (NASDAQ:BOBJ) fit in? With the aquisition SAP now gets the capability to be more than a core business transaction processor. i.e with the intersection of its industry knowledge and the BI knowledge from Business Objects , it can now enter into the relatively new market of Business Health Monitering and Business Benchmarking (not sure if this is a standard term-just coined it). Its future software will probably be able to analyze the business outcomes against indstry norms and suggest remediation (example - Your billing cycle is 10days while in your industry it is 4 days , you can fix it by doing X Y and Z). OR it might be able to proactively moniter the health of your business taking the Enterprise view in context (Example - For the insurance carrier reserve level for the Auto LOB has gone below the norm of X , but the Claims ratio in Commercial LOB is looking good so overall reserve level should not change for the enterprise).

SAP gets a chance to deepen the relationships it has with its customers by adding the business outcomes offering and broaden by penetrating into the Business Objects Legacy contacts.

I expect the Business Objects aquisition to have the effect of 1+1 = 4. The deal is not accretive and on strictly financial terms dilutes EPS and growth rate and hence the resentment from Wall Street on the deal. But then Wall Street was never expected to be a thought leader in how the markets are going to evolve.

I also think that Business Objects and SAP are a good cultural fit. Business Objects and SAP are both leading edge Web2.0 players in their areas (Refer here and here) and hence understand the overall direction of the consumer.