Tonight as I watched the election returns on CNN, I wasn't sure what was more amazing: Obama's historic victory, or CNN's technology.  CNN was able to display up-to-the minute results of each state's elections simply at the touch of their news anchor onto the screen of an election-reporting system.  The anchor could touch a state, then touch a metric, such as various demographics, and instantly cut the election results up by age, exit poll answers, or racial composition.  It blew me away.

But it also reminded me of trying to manage a complex set of application deployments into the Cloud - a virtual private data center.    When you take into account that a reasonably complex multi-tier application with significant load can consume tens of virtual servers, all of which need to function successfully in a coordinated ballet, you realize you need the kind of information and analytic capabilities that CNN has available, just to tune it and keep it working.  Because of this, we've invested in an amazing remote monitoring package, NimBus, which we provide as a service to our hosted customers as well as other customers.  NimBus allows measurement of pretty much any parameter inside a virtual server or the applications running inside of it, from simple (but important) aspects such as CPU or memory utilization, to more complex metrics like database queries per second, slow query count, or pages served per unit time from a web server.  In addition, NimBus can perform user-experience validation by running synthetic (fake) transactions against an application and reporting what the user experiences in terms of response time and page correctness.   All of this is summarized on a customizable dashboard, much like CNN's election status screen (click to enlarge):

So, armed with this information - and hopefully not overwhelmed with too much information - we (or our customers) can tune and adjust their applications for appropriate cost/performance tradeoffs or diagnose performance or efficiency issues.    It has produced great results for the customers who implemented remote monitoring, improving their application response time and uptime, as well as reducing costs.

However, the road hasn't been easy.  The Cloud, by its very nature, is constantly in flux, mutable.   This presents a contradiction in goals to an organization:  to optimize something, it needs to be stable so you can measure it and make changes; yet to get the best economies out of the cloud, you need your infrastructure to be elastic, scaling on demand.  Because servers can come and go, and IP addresses can change, setting up a monitoring system and keeping it running isn't easy.  How can you monitor Apache server #2 if it is only instantiated when the web site's load is too high for one Apache?   Luckily, most of our clients' deployments don't change radically over the short term, so the monitoring package can be set up and continue to run for quite a while before it needs reconfiguration.  However, for very elastic loads, you need to either observe the results of your cloud deployment instead of its internals (such as by snooping on its communications with customers) or have your automatic instance deployments also request on-demand monitoring.

Once you add monitoring to your cloud deployment, you can start to take advantage of the powerful capabilities of Total Quality Management, a management philosophy popularized by W. Edwards Deming.  A core principle of TQM is CPI or continuous process improvement, summarized with the following chart:

TQM says you want to set goals for your process (in this case your software deployment), then you want to run the process (deploy the software), measure the results against the goals, and adjust the settings based on the goals to control the process to produce the desired results (typically a satisfy SLA in the software deployment world.)  However, the real power comes when you report on the results of this process and then use it to take another look at your goals.  The result is continuous improvements in "quality" - in other words, in your ability to deliver the results of your process successfully.

This is how we use monitoring to get the most out of Cloud deployments.

But then I had this insight: why do us - humans - have to be in the loop at all with respect to acting on the monitoring?   Naturally, if the monitoring detects some sort of application or hardware failure, humans need to get involved.  But are humans really necessary for maintaining SLAs?   In today's cloud deployments, especially with systems like Amazon's EC2, the users' application is responsible for both measuring and taking action on application performance issues.   This complicates deployment and coding, as well as tying your application to a particular cloud provider.  However, I believe that the next generation of cloud deployment frameworks will be able to do this automatically, by integrating general-purpose monitoring applications with policy-based cloud management engines.   At ENKI, using our monitoring services, we are already able to automate some of this policy-based management without the need for the application to be aware of the details of this process.   However, a quick caution is in order: if the application isn't designed from the ground up to be elastic (for example, to have new web servers added dynamically) then all the automation in the world won't allow it to participate in automated SLA assurance.