Measuring Clouds: Managing Efficiency and Expectations

One of the things humans like to do (we’ll most of them) is to measure things.  We like to know if we’re better off after we do something.  If so, we do more, if not, we’ll likely do less.  This same process that has plagued mankind continues to plague IT managers over whether their investment in IT is a good business value.  The question is, “How do we measure our efficiency and balance that measurement against our expectations?”

Jerry Cuomo recently posted on this topic at IBMDeveloperWorks in an article called “Measuring the Performance of your Cloud.”   One of my current projects at IBM is optimizing WebSphere Application Server in a variety of environments.  Most notably has been focused on deploying WebSphere onto a cloud over the past few years and now its deploying AND optimizing WebSphere in virtualized environments.  But how do you measure if something is optimized or not?  In virtual environments its more about efficiency and expectations.

In his article, Jerry talked about multiple dimensions to measuring a cloud.

Its interesting to consider the shift from performance being a feeds and speeds discussion to more of one around expectations. Different people have different expectations and I think the list you outlined helps to cover the list of expectations from several perspectives.

His dimensions are:

I have some comments on the dimensions below:

Time to Deploy Interested parties: Developers

Time to get new solutions up and running.  This generally means from OS deployment to application running.   This can have a dramatic improvement on developer’s productivity as well as efficiency of other interested parties (like the guys that have to scale a workload)

Density Interested parties: CIOs, Operations Mgmt, CTOs,etc.

The metric can be applied to a number of objects.  For instance, it could be the number of OS instances hosted on a physical machine, number of processes per VM, users per application, etc.  Cloud providers are also very interested in this metric as its a good indication of the value being derived from IT resources.

Elastic Scale Interested Parties: Line of Business Owners, Cloud Providers and  Operations Mgmt

Elastic Scale refers to the property used to bring resource to service an application (or set of applications).  Its an indicator of how quickly the movement of resource can be achieved.

Line of Business Owners are people who sponsored the development of the application and most likely pay for the hosting services.  They want to make sure that the application is available and can scale to the needs of their users / customers.

Just like Line of Business Owners, Cloud Providers are interested in Elastic Scale as it is a differentiator they can use to show superior infrastructure and services.  Cloud Providers can also be private cloud providers like existing IT shops are today).

Resiliency (Security/Isolation) Interested Parties: who doesn’t want good solid isolation and security?

This becomes somewhat of a tradeoff as we need to measure the risk of high multi-tenant Platforms (running lots of users in the same VM / JVM) as opposed to high multi-tenancy via virtualization of many OS instances. There won’t be one right answer for all.

This is the category where people need to most explicitly challenge reliability, raw performance and flexibility.

Runtime Performance Interested Parties: Everyone

Although this one gets more complicated. Traditionally this drove a lot of discussion about speed but today, with heavy virtualization, to support a highly multi-tenant environment, we will need to start ordering which priorities mean more to us than others.
Time to Genesis Interested Parties: Cloud / Infrastructure providers

Time to Genesis is the time it takes to bring new Cloud resources online.  This includes not only the time to setup a new environment, but also the ability to scale the environment by adding more hardware (compute, net or storage) seamlessly.


Measuring people’s expectations and the infrastructure efficiency will become more challenging, not easier, as time goes on.  Cloud delivers an unparalleled access to dynamic environments, new services and features.  It almost feels limitless.  However, all IT budgets are constrained and driven by service levels for the business.  Many businesses will continue to build their own “private clouds” and they’ll also learn new ways to measure their efficiency against their expectations.

Did you enjoy this post? Why not leave a comment below and continue the conversation, or subscribe to my feed and get articles like this delivered automatically to your feed reader.


No comments yet.

Leave a comment