Realtime Analytics for the rest of us

My two cents added to the discussions here and here.

John's points about the DW purchase decision being pushed to the SaaS vendor and being less relevant to the enterprise (analytic application mid-market customer) is the key to the big switch. A similar analogy would be that when an Enterprise gets its electricity from the electrical company (the GRID ). All that it cares about is the SLA – Can the supplier meet my XXX Megawatt per hour demands at peak loads of YY, 24/7 and after that the pricing is the decision point. What kind of generators the electrical company uses (Hydro , coal , windmills) is an important decision for the electrical company but certainly not relevant to the Enterprise using the electricity.

So this analogy leads to a bit of soul-searching for DW appliance vendors like Terradata , Greenplum etc. “Who is their customer?” are they taking up the role of GE (who manufactures turbines , windmills etc) to serve the SaaS vendors OR do they want to be offering solutions at a level higher to the end consumer that ultimately end up using their appliances. I think a little bit of both and mixture of a lot more partnerships is probably what is going to happen.

It is also interesting to note two other trends that will shape the BI world.

1. Fragmenting of DB market to specialized Database
   
2. Availability of webscale level specialized Databases like BigTable , Hadoop/Hbase at very low entry points.

These trends will lead to a development of a longtail-type market for real-time analytics in an SaaS model (example – Recommendation service based on collaborative filtering, using Predictive modeling results during the underwriting workflow for approving a quote).The reason this will happen is because

a) These kinds of applications are more focused and can be performed in silos’. The whole concept widgets moving at the next layer of functionality and reuse.

b) They are also better served by vendors whose livelyhood depends on bettering the algorithms that power the analytics engines.

c) With things like Bigtable , Hadoop/Hbase exposed to the world at a very low entry point , all it takes is one guy to improve an algorithm and expose to the world as a service.

Update: May 22 ,2008 : I ran into one more Cloud computing Data analytics solutions which is another Column oriented database check it out at Vertica