Cloud computing : Its not just about the infrastructure

So what is the value of Cloud Computing to me(someone who is not an infrastructure guy) ?

The important shift in my opinion is the change that will take place in the programming model for Enterprise Apps as a result of this new paradigm. The SDLC is going through a major change from Develop - Test - Deploy - Release as time/people sliced functions to Develop-Test-Deploy-Beta-develop-test-deploy-Beta2 to functions within the scope of developer responsiblities.

Imagine having your dev/test/prod environments in the cloud all driven through eclipse/netbeans and the movement between the environment's driven through course gained services like AWS webservices API that interact with your hardware to provision the application.

Imagine a library of widgets/services that will make into the Enterprise just has open source made its way in.It is really the re-birth of the Component Based Software taken to the next level.

Imagine treating things like -webscale computing , massive parallel processing- hadoop , specialized analytic capabilities , etc as commodities that can be purchased at any point of time for an application.

Imagine having a large number of small software vendors that thrive on delivering small specialized services/widgets to whom infrastructure is an operating expense and not a capital expenditure.

The above to me is the big switch. Besides the aspects of driving efficiency in infrastructure as a result of scale , the could computing paradigm will enable the building of a large number of software services players that will ultimately compete with the Application development department in the Enterprise.

Note: the link to Eric Schmidt's Bear sterns conference as an interesting observation of -"there will be small number of big players and a large numbers of small players in the cloud computing space"

Realtime Analytics for the rest of us

My two cents added to the discussions here and here.

John's points about the DW purchase decision being pushed to the SaaS vendor and being less relevant to the enterprise (analytic application mid-market customer) is the key to the big switch. A similar analogy would be that when an Enterprise gets its electricity from the electrical company (the GRID ). All that it cares about is the SLA – Can the supplier meet my XXX Megawatt per hour demands at peak loads of YY, 24/7 and after that the pricing is the decision point. What kind of generators the electrical company uses (Hydro , coal , windmills) is an important decision for the electrical company but certainly not relevant to the Enterprise using the electricity.

So this analogy leads to a bit of soul-searching for DW appliance vendors like Terradata , Greenplum etc. “Who is their customer?” are they taking up the role of GE (who manufactures turbines , windmills etc) to serve the SaaS vendors OR do they want to be offering solutions at a level higher to the end consumer that ultimately end up using their appliances. I think a little bit of both and mixture of a lot more partnerships is probably what is going to happen.

It is also interesting to note two other trends that will shape the BI world.

1. Fragmenting of DB market to specialized Database
   
2. Availability of webscale level specialized Databases like BigTable , Hadoop/Hbase at very low entry points.

These trends will lead to a development of a longtail-type market for real-time analytics in an SaaS model (example – Recommendation service based on collaborative filtering, using Predictive modeling results during the underwriting workflow for approving a quote).The reason this will happen is because

a) These kinds of applications are more focused and can be performed in silos’. The whole concept widgets moving at the next layer of functionality and reuse.

b) They are also better served by vendors whose livelyhood depends on bettering the algorithms that power the analytics engines.

c) With things like Bigtable , Hadoop/Hbase exposed to the world at a very low entry point , all it takes is one guy to improve an algorithm and expose to the world as a service.

Update: May 22 ,2008 : I ran into one more Cloud computing Data analytics solutions which is another Column oriented database check it out at Vertica

Cloud computing and Datawarehousing

Following my last blog entry , my brain continued elaborating on the thought of Cloud Computing Adoption in the Enterprise.

Having worked in the Enterprise Space for So long , I am hard pressed to come to terms with the notion that Enterprises will be willing to completely outsource their Information Management and IT infrastructure and more so in a constrained environment like that of Google AppEngine.

Amazon AWS with its ala carte is still a better option to Enterprises as compared to Google App Engine. As you can pick an choose what you want. I think the key is that any Cloud computing vendor needs to "FIT IN" into the Enterprise's Architecture . This basically imply's that more entry points you have to the cloud infrastructure the more use-cases you will have for Enterprise Adoption. So , it seems like Amazon has a better strategy for Enterprise Adoption. Another use-case for Enterprise Adoption is via an SaaS vendor case in point - Vertica . Vertica is a user of cloud infrastructure from Amazon. But the way the cloud is coming into the Enterprise Architecture is via an SaaS vendor.


Another use-case that came to my mind is the impact to the EDW world. With things like BigTable and simpleDB exposed , why would an Enterprise invest in highend Databases like TerrrData - why not use a proven scalable platform like BigTable to run your analytics. In any case you need to do your EDW work in house on separate machines from your core systems - So using an on-demand infrastructure for such needs makes sense.

Shame on you if you cannot start a .com now !

Life could not have been better for developers. I thought that what Amazon had done with AWS was the best developers could have had , But now we have one more entry into the space-- Google App Engine

It is interesting to see how the SaaS/HaaS space is shaping up. Googles App Engine seems like a layer above AWS. i.e a hosted development platform - the toolset for developing an application on App Engine will be focussed and consequently also limiting. App engine provides excellent integration to google's services(like single signon etc) but it is also limiting , i.e I cannot use Java (at least right now).

AWS on the other hand gives you more power (build your own machine the way you want it) but it has a higher learning curve and if you are running a shop on EC2 you will require a System Administrator to manage your website.

While at this nascent stage of Cloud Computing it seems obvious to compare AWS and Google's App Engine but I think that as the market evolves both the offerings will really address separate markets. Google will get its pie from transitioning the traditional Rapid Application development tool (RAD) customers and Amazon from the traditional IT Infrastructure shops. They certainly converge at some level but it will be years before that happens.

It is also interesting to see that traditional IT Majors like IBM , SUN and HP are missing in action from this revolution. One reason could be that at this point AWS and AppEngine seem like a mom-pop shop offerings(Small business and non mission critical) , how do these offerings translate to Enterprise customers and what are the SLA's that will mature these offerings to Enterprises is something I look forward to understanding in 2008.

Maybe Google and AWS needs someone like Capgemini or Accenture to do this.

If you have the time check out the video on google app engine

The Long Tail - Taken to the extreme

OK , I have read the long tail and I agree it is a great book. It is very interesting to see how Amazon is taking the concepts of Long tail to the extreme.

Refer : AWS fulfillment service : is a service where you can use the Amazon warehouse to store your products and they will then manage the shipment , order execution and payments for the item.

Earlier , when AWS-EC2, S3 got introduced I really thought that it was vertical integration. But now I am starting to believe that what Amazon is doing is just scaling up every core competency they have. This is not really vertical integration in the sense that it is not coming from strategic business decision of getting into a market, but really coming from the fact of making the overall cost model of operations better. Volume drives costs down and if Amazon can manage to get more shipments there cost per shipment will go down. I don’t thing that they are getting into the warehousing business big time (i.e taking shipments from containers from china etc) , But seems like a experiment that might / might not work.

The key thing in Long Tail economics really boils down to - what is the incremental cost of exposing an Enterprise's inner business functions to the world as a competency. Frankly if you need to re-architect your processes to support this you need to rethink on adopting Long Tail concepts into your business.

Database 2.0

For a long time (decades I believe) it seemed like the General purpose Database was the silver bullet. Object Oriented Databases tried to take the shine away from RDBMS , but only to strengthen the notion that RDBM’S do best at what they are supposed to do – Store , Query and Update Data.

The application development world adjusted to the notion that the Object Relational mismatch is a reality and instead of fighting it let us work with the RDBMS systems and make world a better place. Tools like Hibernate had great success because they accepted the strengths of RDBMS as compared to competing technologies at that time like EJB's . The Enterprise was also caught up in the ERP wave where everything in the Enterprise needed to have a predefined structure, vocabulary established etc.

2007 brought some interesting changes in the DB world. Web 2.0 brought the concepts of self organization , realization that all information does not reside in-house , that everything cannot be structured , that Information Management is not just EDW / BI - there is a whole world out there with unstructured information and Semantic Web . Consequently SQL and RDBMS is not the silver bullet.

2007 also brought some interesting DB’s to the forefront including SimpleDB , BigTable , HP Neoview , CouchDB each of these serving very different purposes finetuned for particular needs.

So the definition of Database 2.o in my opinion is that it is the realization that all information in the Enterprise cannot be out into RDBMS , cannot be structured and that there can and will be multiple datasources to information within and Enterprise.

So what is my point ?

I think the Database market is splitting into two layers
1. The general purpose database market - now turning out to be a commodity market
2. The specialized DB market

So finally I would like to close on the trigger that got me to think the above - The mysql acquisition by sun – Why does sun need a database product NOW ? it has severe wall street issues and convincing the market of buyout of a company that gives away free products is going to amazingly difficult.

Here is the answer I came up with .
With the technology and engineering capability needed to build a general purpose database generally available – Building a high performance RDBMS system that competes with the likes of Oracle and DB2 does not seem like a very big challenge. So if competing on price and brand are your differentiator's in the general purpose DB market then what better way to compete by giving the product free.

Now monetization on mysql and revenue potential is a separate discussion. My discussion above is keeping the number of deployments in mind.

more reasons for the Sun / mqsql acquisition

Struts , JSF , spring mvc , GWT - what is it going to be

I was reading the blog at java based web frameworks for 2008 and ended up doing a query on indeed.com.

So I compared the following
1. Struts
2. JSF (Java server faces)
3. GWT (Googles web application toolkit)
4. Spring MVC

Here is what I found

And Now comparing the Relative growth

As I had expected Struts is the clear leader and my guess just like Mainframe's still are a very important part of the Enterprise , Struts will always be there in the web 1.0 world.

JSF - to me is really a web1.0 technology -just an upgrade to struts.Sun has a knack of making things complicated and I dont think it has handled JSF any different than EJB's 1.0.

Spring MVC - has an interesting story, Spring Core(the middleware piece) has a very strong acceptance and spring core has a bigger audience than spring mvc. It's simplicity of programming model , well thought feature's make it a good candidate for use.

GWT in 2008 is IMO going to be the real winner. About an year ago I had posted that 2007 would be the year for GWT . Well looking at the numbers I think it was. It has not overtaken struts but the relative growth has been the highest - 30,000 % WOW!.

It has three things going for it
1. Java based (so the corporate java developers have an upgrade path)
2. It exposes a whole new capability of ground up AJAX applications not available in Struts , JSF or spring MVC
3. Simple (in every way - learning , developing , building , deploying)

I ran into a client recently who was using GWT very aggressively , Spring core and Spring MVC.When I asked him about why he was using Spring MVC and GWT- he commented the following

- Everything on website cannot be AJAX based application style. I will have some pages that really are suited for html (jsp) type content . I am using Spring MVC for that(example login page , help pages )
-Also Spring MVC is also a good place to put everything together. It works well with J2EE file structures etc and a great way to launch everything and coordinate the website.

The above view is what I think will shape 2008.

Google Energy and Vertical Integration

Google’s(NASDAQ:GOOG) announcement to enter the Energy space kinda surprised me but when you look under the hood there seems to be very good business justification for this.

Google is doing what Henry Ford did in the auto industry and Rockefeller did in the Oil industry – Vertical Integration.

Vertical Integration is defined as - The ownership by the same company of different functions in a supply chain relating to the provision of a particular good or service. Vertical Integration is generally done to lower the transaction costs in the supply chain and synchronize the supply and demand across the chain. Vertical Integration was pioneered by business leaders like Rockefeller and Henry Ford , where after reaching a certain size in their business they expanded into owning the supply chain so as to minimize the risk , control quality and lower transaction costs. Rockefeller after owning oil refinery branched out into oil distribution, oil retail and oil production.

Electricity costs for any SaaS vendor ( Google included) Electricity costs are significant and it is only going to get worse with Oil at $100 and the Global warming problems with coal. So it makes sense to manage the risk on the supply side and try to solve the problem. Google with its core competency of managing and monetizing innovation is also well placed to take up the challenge.Besides in my opinion Energy is going to be next frontier of innovation and to keep on sustaining at PE multiples of 50's long term GOOG needs to be looking at the next growth industries.

What got me thinking on reading about the Google Energy post is the approach Enteprise2.0 companies(like Amazon and Google) are taking towards Vertical integration. Thought leaders like Don Tapscott have argued that in the 2.0 world the transaction costs (the costs for collaboration) that justified vertical integration have evaporated to almost nothing now and vertical integration potentially does not make sense now as the integration costs internally in the Enterprise could be more than taking the route of ideagoras or Crowdsourcing

Google’s Energy entry and Amazon’s Webservices business in my opinion are Aspirational Core Competency based Vertical Integration transactions.

For Google:

Aspirational –Energy is the next thing
Core Competency - Managing and monetizing innovation
Vertical Integration – improve downstream supply chain.

For Amazon:

Aspirational - Become an HaaS Vendor
Core Competency - Software Engg and Systems Management
Vertical Integration - improve downstream supply chain.

Update : dec 29th: Here is another example for Googles vertical integration strategy. Also interesting to note that Google has been integrating downstream and not a lot upstream ( Android) , it did not not end up launching a GPhone as people had predicted.

Business IT Alignment and SOA governance

Congratulation Bryan Mjaanes on successfully implementing SOA program at Zurich Insurance .

There was one slide that I really liked that in my opinion captured the Business IT Alignment problem quite well.




The diagram explains quite beautifully how the information flow and feedback mechanism's work for SOA governance. The link from Enterprise Architecture / SOA to Business Process in my view represents the two constituents in the Enterprise that need to be aligned and in my experience this link is generally the weakest link in the picture. What I have seen in Failed SOA cases is that the shadow IT(part of Business) will end up using SOA tools and technologies (webservices etc) and solve their business problem and keep on shouting to the world that they implemented an SOA architecture. See diagram below , This generally happens when the SOA program does not have a senior management buy in and the Enterprise Architecture group is really a figure head and not very powerful.

Subprime meltdown in layman's terms

Want to explain the subprime problem to your Dad without boring him??

Funny Video :) Enjoy!