This posting is a summary of Lesson 3 in the Cloud 101 Class, in which different methods of allocating cloud computing resources to meet customer demand were discussed, and how those methods affected costs.  You can return to the first posting in Lesson 3 here.

• There are three major ways that vendors allow customers to request resources from the cloud: fixed instance size allocation, quantized instance size allocation, and variable instance size allocation. Each method allows customers to request computing resources according to their needs, which potentially offers significant cost savings compared to purchasing or leasing computer hardware.
• Some vendors use a fourth method in which instance size varies based on the number they pack onto their servers, which does not offer the guaranteed resources that people expect from the cloud, though it is often offered at a fixed price. This is more of a 'hosting' offering than true cloud computing.
• Fixed instance allocation such as that used by Amazon EC2 offers low instance cost, but encourages customers to over-buy instance sizes to avoid performance problems, which increases the actual cost to the customer. Responding to changes in demand requires allocating new instances, which is complex and costly. Third party vendors have arisen to solve some of these problems.
• Quantized instance allocation solves some of the problems with fixed instance allocation by giving customers more flexibility in selecting instance sizes.
• Variable instance allocation allows customers to request exactly the resources they need from the cloud, but requires them to know exactly what those requirements are. Most customers are not familiar with the techniques necessary to understand their true requirements.
• Each allocation method requires customers to understand how much resources their applications need at a far more detailed level than ever before, potentially creating the risk of performance problems or unexpected charges.
• Automatic scaling, either by adding instances or by resizing existing ones, can simplify the requirement of understanding how much resources an application needs to run successfully.  However, it can result in unpredictable bills.
• Because hardware is always changing, cloud vendors may not be able to guarantee an exact performance level per unit of cost. How serious this problem is depends on the policies of the vendor, which may not be transparent to the customer.
• Cloud vendors have to keep unused inventory of servers available for peak demand, which can increase costs. The exact amount of these costs is not generally known.
• Total Cost of Deployment, the actual cost the customer must pay to run their application, is difficult to calculate for cloud deployments (though simpler than TCD calculations for building your own datacenter.) This difficulty threatens the success of cloud computing, since customers are looking a simple way to understand their costs, not just find the lowest costs.

Lesson 3, "Behind The Scenes in Determining the Costs of Cloud Computing" continues with this discussion of how unused capacity affects the cost of cloud computing.  This lesson builds on the previous ones.   Please see the overview in Lesson 3.1 if you have not already read it. 

So far, we’ve discussed how the cost to the cloud computing vendor is affected by how their application instances are allocated on the vendor’s servers. However, there is an even larger factor that affects the vendor’s costs, which is how many unused servers they keep in their inventory. For example Amazon EC2 rose to fame with their now-famous example of how they digitized nearly 100 years of microfiches using 100 virtual servers in one day. Now, customers routinely use EC2 to provide thousands of machines for short periods to perform medical or scientific calculations. However to provide these burst of computing, Amazon has to keep a large inventory of unused servers around, by some estimates as much as 66% of their total on the average. This of course adds greatly to their costs for data center space, equipment cost, and repairs, even if the servers are turned off when not in use. This isn’t generally discussed by cloud providers, but if you examine the quarterly reports of the cloud providers that are public companies, you may be able to see some interesting facts.  One would expect that over time, these costs would have to be passed on to the customers, even if the vendor keeps the prices low for a while to buy market share.   In other words, you can expect cloud computing costs to rise over time as vendors establish themselves.

While the bulk of business and web computing varies in load over time, the variation is typically constrained and in the range of two to three times the minimum usage, allowing for an average load across many customers that varies far less than what Amazon must build their datacenters to accommodate. If a cloud vendor is willing to give up on taking business from customers who want thousands of servers for short periods of time, they should be able to manage their costs by keeping a much smaller inventory of unused equipment in stock, and in turn passing the savings on to their customers.

For vendors that choose to supporty widely varying loads, the latest "innovation" is allowing customers to reserve capacity so that they can be sure the computer power will be there when they need it (which Amazon doesn't guarantee, otherwise.)  Especially in the case of prepaid reservations, this allows the cloud vendor to get help from the customers in purchasing hardware which otherwise would simply be kept around at a loss.

Lesson 3, "Behind The Scenes in Determining the Costs of Cloud Computing" continues with this discussion of a method of allocating cloud computing resources to customers in adjustable, granular chunks.  Please see the overview in Lesson 3.1 if you have not already read it.

Variable Resource Allocation

With this packing strategy, the vendor allows the customer to select whatever values for CPU and memory allocation that they desire.   In our housing analogy, this corresponds to the developer allowing families to have the most efficient housing allocation by having their houses built to specification, with any number of bedrooms and baths.  This creates a packing problem for the vendor because the instances have sizes that do not evenly fit into their servers, leaving lots of fragmentation.

To solve the fragmentation problem, vendors using variable resource allocation buy very large servers with lots of CPU and memory, so that the fragmentation is a small percentage of the overall server size, and hence cost.  In addition, some vendors allow on-the-fly resizing of instances, which in our housing analogy corresponds to allowing families to remodel their houses to add rooms, or even subtract them.  This truly fulfills the "pay for what you use" promise of cloud computing from the customer's point of view.   This technology also benefits the vendor in reducing fragmentation because it is based on the ability to move instances around from server to server easily (often without disrupting the customer's running software) to coalesce free server resources so that they can be sold to other customers.   The ability to resize instances also means that the instances are persistent, meaning that the state of the instance (including the loaded software and stored data) is saved even if the instance is turned off, resized, or the underlying hardware fails.  Instance persistence offers significant benefits to customers since the software need not be written to take into account dissolution of the instance, and additionally, vendors can offer uptime guarantees for the instance.

This packing strategy moves the fragmentation from inside the customers' instances (where they are paying for it) to the vendors' servers, where they are generally better able to manage it by packing other customers' instances into the unused spaces.

Once a vendor offers variable resource allocation, monitoring the customers' usage and then adjusting the resources allocated to the instance(s) automatically becomes a possibility, which can reduce costs dramatically for highly variable computing loads, as well as ensure the performance of the application if the demands on it spike.

Advantages of Variable Resource Allocation

-        Flexibility.  Customers can allocate exactly the resources that their software applications need. In addition, if the vendor supports on-the-fly resizing (often called scaling), customers can adjust resource allocation to optimize performance or payments as computing loads change.

-        Ease of Use.  Variable Resource Allocation often is offered with instance persistence, allowing customers to write software without worrying about the reliability of the underlying cloud platform.

-        Cost.  Since this is truly pay-as-you-go computing, the customer can optimize their overall costs.  By allowing the customer to remove unused resources from their instances, the fragmentation in the cloud is moved from inside the customers' instances into the vendors' servers, where they are generally better able to deal with it.  Automated scaling can save customers with highly variable computing loads large amounts on their computing bill.  As an example, most physical datacenters run at between 2-25% utilization, so automated scaling can return resource savings of 75-98%!

-        Easy to Understand.  Since customers get exactly the resources they request, pricing can be simplified to a simple rate per resource used, like dollars per CPU-hour.  This can be an advantage as well as a disadvantage, see below.

Disadvantages of Variable Resource Allocation

-        Hard to understand.  Customers often are not used to variable resource allocation, in which they become fully responsible for knowing how much resources their application actually needs to do its job, since a bad guess can result in an application crash.  This is a big change from the old non-cloud model where they would simply buy much more server than they needed and not worry about resource allocation, or even the fixed or quantized cloud packing models, where most customers still do the same.  This also makes it difficult for them to predict what their costs will be since those costs now depend on what is going on inside their software and on end-user activity, instead of some fixed amount that they are paying.  This problem is particularly bad for startup companies, who have no historic record of demand or software efficiency in meeting it.  Automatic scaling can eliminate the need to understand what resources the software needs by eliminating the need to predict appropriate instance sizing, but it still doesn't help to predict costs.

-        Cost.   Much like any scheme in which resources are allocated on demand, auto-scaling can surprise the customer with a large resource bill, so limits should be placed on its ability to increase instance resources. There is also a potential for slightly higher costs to the vendor due to increased resource fragmentation on their servers, which may be passed on the customer. Overall, the vendor is better able to manage fragmentation than the customers are, so this additional cost should be negligible or perhaps not even an issue.  The vendors' charges are generally straightforward since they are offered as a resource hourly rate, but calculating TCD with this allocation method is still difficult, mostly because customers have difficult calculating their resources requirements. 

Best Customer Match for Variable Instance Allocation

-          Customers who need to manage internal fragmentation to save money, and understand the resource usage profile of their application.

-          Customers who need to manage internal fragmentation, and have chosen a vendor with automatic resource scaling so that they don't need to understand their application's resource needs over time.

-          Customers who can save significantly on cost by taking advantage of automatic resource scaling (if the vendor offers it.)

-          Customers who wish to transfer a physical datacenter to the cloud without having to worry about sizing the resources correctly, or re-architecting their applications to accommodate fixed or quantized instance allocations

Lesson 3, "Behind The Scenes in Determining the Costs of Cloud Computing" continues with this discussion of a method of allocating cloud computing resources to customers in adjustable, granular chunks.  Please see the overview in Lesson 3.1 if you have not already read it.

Quantized Instance Allocation

Some cloud vendors have chosen to solve the problems of fixed instance allocation by creating a build-to-order service, where customers can order instances based on their actual needs, with some restrictions.  To make this packing strategy efficient, customers are only given the option of choosing either CPU or memory size in fixed increments, with the other being set by the vendor based on a formula.  In our analogy, this is the equivalent of the builder of a housing development offering custom-built homes, in which you may choose any number of  bedrooms, and you then get a proportional number of bathrooms, like for example ½ bath for every two bedrooms.  An example of a provider that uses quantized instance allocation is GoGrid.  This is a variation on fixed instance size allocation.

In effect, this packing strategy is like the fixed instance size strategy, but with many more options.  However, because there are more possible instance sizes, they don't fit into the vendor's servers as efficiently as a smaller set of instance sizes, increasing fragmentation.   Many new cloud computing services coming into the market use quantized instance allocation, perhaps to resolve some of the issues with fixed instance allocation as used by Amazon so that the new vendors can compete effectively.

Advantages of Quantized Instance Allocation

-        Flexibility.  Customers are offered more options for instance sizes so that they can choose the right ones for their needs, saving on costs.

-        Efficiency.  Packing is still relatively efficient if the cloud system does some optimization, so resource savings from this packing strategy can be passed on to customers.

-        Somewhat easy to understand.  Though not as simple to understand as Fixed Instance Size Allocation, customers generally understand this model well, and like the idea of sizing instances to their needs.

-        Cost.  Because vendors often offer more choice in instance sizes, customers may potentially save over fixed instance allocation because they don't need to order as many unused resources.

Disadvantages of Quantized Instance Allocation

-        Customers' needs not exactly met.  Limited and fixed selection of ratios between CPU and memory allocation means that customers' needs are not exactly met, resulting in overpayment to get the necessary resources, or inadequate performance.

-        Inflexibility.  Again, due to current technology limitations, and limitations of this packing strategy, instance sizes cannot generally be changed while the customers' application is running.  To solve this problem, customers have to use the same scaling methods as for fixed instance packing strategies.  Some cloud vendors assist with the necessary instance moves or load distribution, somewhat circumventing this problem.  However, the application software generally must be written so that it understands the scaling method, or it will fail if it is moved or duplicated.  Also, because of the fixed ratio between CPU and memory, customers have to buy more of the selectable resource than they need, if they need more of the dependent one.

-        Complexity.  Customers may have difficulty understanding the fixed relationship between CPU and memory that this allocation strategy requires, leading to potential frustration with the cloud vendor.

-        Cost.  Inefficiencies in the packing strategy as well as potential fees for services designed to mitigate its inefficiency and inflexibility can raise the cost of this type of service.   Calculating TCD is difficult.

Best Customer Match for Quantized Instance Allocation

-          Customers who need more flexibility than fixed instance allocation, or want to optimize their internal fragmentation.

-          Customers who are highly technical and want to administer their own instances and take responsibility for instance size management.

-          Customers who don't have applications that vary significantly in their resource usage over time.

Lesson 3, "Behind The Scenes in Determining the Costs of Cloud Computing" continues with this discussion of a method of allocating cloud computing resources to customers in fixed-size chunks.  Please see the overview in Lesson 3.1 if you have not already read it.

Fixed Instance Size Allocation

This strategy is the one that existing cloud computing customers may be the most familiar with, since it was chosen by Amazon EC2, one of the pioneers in cloud computing.   Originally, Amazon offered a small selection of virtual computer configurations, or "instances": small, medium, and large.  Each one offered a fixed ratio of CPU to memory.  As customers started complaining that these ratios didn't meet their needs, Amazon divided these instances into high-CPU and high-Memory instances which offered more of the type of resource that the customer needed more of.  Currently, they offer 5 different instance sizes.  In our analogy this corresponds to a housing developer creating a limited set of house plans for a housing development: customers have to choose from a small menu of bedroom/bathroom count combinations that best meet their needs.

Amazon took a very analytic approach to finding an optimal packing strategy.   Since their instance sizes have fixed ratios of RAM to CPU, and a limited number of choices for the quantity of each, they can pack customers' applications into their servers with nearly 100% efficiency.   The problem becomes much like solving a preschool-level jigsaw puzzle, in which the pieces are pre-designed to easily fit into a nice, rectangular whole. 

In the graphic, the large rectangle represents the server's resources (CPU and Memory), and the blue spaces represent unused resources. When a customer requests an instance, the Amazon system simply looks for a "hole" (free resources, shown in blue) in the jigsaw puzzle of one of their servers that fits the size of the "piece" (the new instance, represented by a colored rectangle).  Because the instances are all proportional to the ratio of CPU and memory built into the servers (though there may be more than one type of server depending on the variations in the offered instance types such as Amazon's High Memory Instances and High CPU Instances), the instances fit into the servers with very little unused space (called fragmentation) since new instances can always be placed into the empty spaces, so this is an efficient packing strategy.  However, the fixed size of the instances means that customers often have unused resources inside their instances since they have to always buy extra space to allow for peak or increasing load, memory leaks, or other growth in their application requirements.

Advantages of Fixed Instance Allocation

-        Low cost per instance.  The vendor can keep their servers busy with very little server fragmentation so they are used efficiently, reducing their costs and supposedly the prices that they charge customers (as we shall see in the next lesson, this assumption is not exactly true.)

-        It is simple.  Customers can easily understand what they are getting and what they will be paying for.

Disadvantages of Fixed Instance Allocation

Unfortunately, this strategy optimizes convenience and efficiency for vendor, not for the customer.

-        Doesn't meet customers' needs without additional effort.  Much like in the years before cloud computing, where customers had to buy a physical server and then use as much of it as they needed, customers have to deal with the instance either being too small or too large for their needs.  This packing strategy moves fragmentation in the server as whole into each customers' virtual server (where they are paying for the unused resources.)

-        Complexity.  Because of limitations in the Xen virtualization system used by Amazon and many price-focused cloud vendors today, instance sizes cannot be changed on-the-fly to accommodate changes in demand.  Also, changing them while in use would destroy the efficiency of the packing strategy, much like the example with oranges of different sizes.  To solve this problem, customers have two choices, both of which cost them money.  One choice is to buy a larger instance than they need to allow for future increases in server loading, which results in paying for more than they need.  Or, the customer must use the vendor's programming interface to get more resources.  This can be done in two ways, by either moving their programs and data to larger instance when their application comes close to reaching the limits of the instance it is deployed on, or they can design their software to distribute the computing load across multiple instances and start a new one when their existing instances cannot accommodate the load.  Moving the application from one size instance to another, like moving from one house to another, is a complex process that requires either people to detect the problem and solve it manually, or other instances to detect the need for the movement and execute the steps to do so.   Similarly, adding new instances to shoulder additional load, like splitting your family into two houses, is a complex process that requires an arbitrator instance or hardware unit (called a load balancer) to send requests for work to multiple instances, as well a as manager instance that detects the need for the additional instances and starts them.   Additionally, the application software generally must be written so that it understands the scaling method, or it will fail if it is moved or duplicated.

-        Cost.  Aside from having to pay for instances that might be too large, all the extra instances that are responsible for responding to changes in loading are, of course, an extra cost to the customer.  To solve the complexity problem, a host of cottage industries or value-added-resellers (VARs) for Amazon EC2 have sprung up that do this for you.  Naturally, they also charge for their services, which include the cost of the Amazon instances necessary to provide them.

-        Analyzing cost is confusing.  Whether these extra costs exceed the savings from the low instance prices that supposedly come from the efficiency of the packing strategy depends on analyses that the customer has to make which can be very, very difficult to do accurately.  Calculating TCD is a graduate-level project that is beyond most customers' capabilities, especially when taking into account additional fees that are often charged by vendors (see Lesson 4, Cloud Pricing Strategies)

Best Customer Match for Fixed Instance Allocation

-          Customers who like the very low prices for small instances.

-          Customers whose loads don't vary much so they don't need to manage unused instance capacity.

-          Customers who are highly technical and are capable of programming around the complexity of managing multiple instances to accommodate growth in demand.

Overview - How do Cloud Providers Pack 'em In

When I speak with potential and current customers of cloud computing, the question I'm asked most often is "how much will it cost"?   That depends primarily on three things: how much of "it" the customer wants, how providing it affects the provider's costs, and of course how the provider does its pricing, which we will cover in another lesson.  The fundamental principle behind all cloud computing is shared infrastructure (computers, network, and storage) to provide computing services to many users.  As we saw in Lesson 2, virtualization is the key to accomplishing this by allowing many customer applications to share a powerful server without being affected by each others' resource usage and by providing security between customers' applications.  But virtualization alone doesn't determine how the provider allocates their infrastructure to each customer.  Instead, what determines the provider's costs is largely their packing strategy.  This lesson will explain why packing strategies are important in determining what a customer pays for cloud computing and how flexible the cloud computing offering is, as well as discussing the most common packing strategies and their advantages and disadvantages.  Finally, the lesson will explain why cloud vendors can never exactly explain the relationship between cloud computing performance and cost.

In this lesson and in the future, we will be talking about Total Cost of Deployment (TCD), which is the actual cost of putting your application into the cloud, versus what a vendor charges or claims to charge.  TCD is ultimately what customers are interested in, but unfortunately, they usually don't have enough information to determine it, being stuck with vendors' per-hour or per-instance charges instead, which may vary greatly from TCD.  Calculating TCD is so complex, that this single obstacle represents a significant threat to the future of cloud computing as  it exists today, since cloud customers are led to believe that by outsourcing the infrastructure, they are also outsourcing the complexity of calculating what it costs.  Many customers may choose to either be less focused on cost and elect to trust that their vendor is using the underlying cloud technologies to give them a fair deal, or they may seek out fixed-cost packages that include a range of resources without requiring an understanding of the costing details.  If this matches the approach you wish to use to relate to your cloud vendor, you can skip the rest of this lesson.

Packing Strategies

To understand why a packing strategy is important, let's start with a simple analogy: packing oranges into a crate.  The number of oranges you can get into a crate depends on how big the crate is, how big the oranges are, and how you pack them in, which is the packing strategy.  If you just throw the oranges in, you they will jumble up and leave lots of free space.  Instead, packing houses place the oranges into the crate in an optimal way such that every empty space is filled.  The obvious parallel to cloud computing is that the crate is a provider's server, and the oranges are customer's software applications.  However, cloud customers' needs aren't exactly like oranges: they vary widely in "size", with some software applications requiring more CPU or memory than others.  Using the regular packing strategy that works for oranges to pack many customers' applications into a single server won't satisfy most customers who want cloud computing to meet their varied needs.

Now that you understand why the packing strategy is important, let's look at an example that more closely matches how cloud providers allocate server resources to customers: placing families in houses.  To satisfy the needs of a family well, a house must have the right number of bedrooms and bathrooms.  Too few of either and people can suffer.  Too many, and they pay too much rent or mortgage.  Here, the bedrooms and bathrooms correspond to CPU power and memory usage in the cloud computing world, and the housing development corresponds to a cloud server.   If we measure the housing efficiency - the efficiency of the packing strategy - of a neighborhood by how well its houses satisfy its homeowners on space and payments, the best efficiency would occur if each family had the number of bedrooms and bathrooms that it needed.   But families come in all different sizes, and to make the problem of efficient packing even harder, they tend to change in size.  If families were forced to live in just a few house designs (sounds like a country I wouldn't want to live in) then pretty much everyone would either be paying too much for their housing or struggling with not enough space and resources.   Instead, we have come up with ways to improve the packing strategy for housing: families can move to another house if theirs no longer efficiently meets their needs, and they can remodel if they want to stay in the same location, or even rent out extra rooms to downsize the space they're paying for.  As we shall see, each of these strategies for optimal packing corresponds to techniques that cloud providers use.

There are three packing strategies that cloud providers use today:

1)      Fixed Instance Size Allocation, in which customers are offered a choice of virtual computer "instances" with pre-allocated quantities of CPU and memory.

2)      Quantized Resource Allocation, in which customers can build their own virtual computer by setting the resource allocations from a limited choice of quantities.

3)      Variable Resource Allocation, in which customers can choose whatever allocation they please.

Each strategy has advantages and disadvantages in terms of resource costs and limitations on flexibility and manageability that are imposed on the customer, though each is in keeping with the basic premise of cloud computing that each customer is guaranteed the resources they ask for.   In the sections of this lesson that follow, we will discuss each one and how it affects what the customer pays.

There is also a fourth packing strategy in which customers are not guaranteed the resources they ask for.  In this strategy, the server's resources are split up using pseudo-virtualization that divides its resources among all the customers' instances depending only on how many instances are loaded on the server.  This can result in very poor performance as well as potentially betraying or cheating the customer, depending on what was promised to them.  On the other hand, this allows vendors to offer services at very low cost, like coach class on a packed airplane.  Many "hosting" providers offer this service under the banner of cloud computing.  But this is not truly cloud computing, because one of the requirements - hardware abstraction, which hides the details of the underlying hardware's capabilities - is not met.   However, it may meet the needs of many customers who need very little resources, such as very small businesses or pre-launch web entrepreneurs.

In 2006, two technologies came together that formed the genesis of Cloud Computing, making real Sun’s motto coined by John Gage: “The Network is the Computer.”   These two technologies are virtualization and of course, the Internet. Both had been around for a while, but in 2006 for the first time, they were ready to start a revolution in how computing was delivered to customers.

Since the core principle of Cloud Computing is delivering services from shared hardware, many people say that it is nothing more than a warmed-over version of the 1970’s concept of “timesharing”, in which a single computer’s power was sliced up in millisecond intervals to server many users.   However, nothing could be farther from the truth. Instead, the advent of virtualization technology allowed creation of virtual computers for each customer inside a single physical server so that from their point of view, they didn’t have to share with anyone. Together with the rapidly increasing power of servers – they run more than 10 times faster today than in 1990 – sharing a server no longer represents  a compromise for most users.

Virtualization is, from the user’s point of view, the capability to carve a computer up into pieces such that the software running on each piece has no way to detect that it is not running on its own dedicated hardware.   This feat of seeming magic is the result of a combination of hardware features in the computer and software systems that abstract (or hide) the physical characteristics of the computer hardware from the running software. By hiding the hardware, it becomes possible to run many software systems on a computer at once without any of them being allowed to interfere with each other or the operation of the hardware, and to move software from one virtualized computer to another without any change. These software systems can be operating systems, such as Windows or Linux, each themselves capable of running many programs.   With proper security, each operating system can serve a different user or group of users, and the computer can then be carved up among completely different users that don’t have any relationship to one another.   For more information on virtualization, go to the Wikipedia page.

If you are familiar with virtualization technology, please feel free to skip this section.

On a physical computer such as the PC on your desk, there are a number of programs running (the Applications), each of which talks to the Operating System, which makes the resources of the underlying hardware (the CPU, memory, and peripherals) available to them through a software system called the “I/O drivers.” Please see the diagram below, which shows this arrangement for a compute used by the Sales Dept.

Depending on how the Operating System is set up, the Applications compete for resources, which is why for example your computer seems to slow down if your antivirus is running. The Operating System protects the programs from accessing each others’ data and crashing each other, but if one of the programs misbehaves and crashes the Operating System, your whole computer stops working. If two different users are using different applications, then the actions of one user can affect the other because they are competing for the hardware resources on the computer.

On a virtualized computer system, each user can be given their own environment, called a “virtual machine” or VM. To that user and their software, the VM “looks” just like a real, physical computer. Each VM is assigned a portion of the computer’s resources that it cannot exceed, and runs its own operating system and applications, as though it was a separate (less powerful) computer. While the programs running in a single VM can interfere with one another, they cannot affect a different VM because they are inside an ironclad “jail” that limits access to the system’s resources as well as to other VMs. This “jail” is provided by a combination of certain hardware features in the CPU and a piece of software called the Hypervisor (hyper-supervisor), a special purpose operating system that creates and maintains the VMs as well as serving their requests for access to hardware resources. Each VM’s operating system, instead of talking to I/O drivers that can access the hardware, talks to “paravirtualized I/O drivers” that work with the hypervisor to give the VM access to the hardware. Much like a security guard, the paravirtualized drivers fool the VM’s operating system into thinking that it has direct control over the hardware, while controlling the flow of data to the computer’s hardware so that the VM’s demands don’t affect other VMs. Together with hardware features in the CPU that prevent one VM from even seeing the memory occupied by the other VMs, this allows for a high level of security between VMs, generally accepted to be as good as could be set up between different physical computers. For more information on hypervisors, go to the Wikipedia page.

The hypervisor is also concerned with management tasks such as creating, destroying, and adjusting the resources (memory, CPU, I/O Bandwidth, etc.) assigned to each VM. These management operations can be initiated from a command line, a user interface, or programmatically through an API (Application Programming Interface). Because of this flexibility it possible to write software that manages an entire datacenter full of VMs from a single terminal, or even allows users/customers to create new VMs for themselves. Hypervisors can typically manage up to tens of virtual machines per physical machine.

One important feature of hypervisors is that instead of requiring the user to create an empty VM and then load the operating system and the application software into it, much as had always been done with physical servers, one could now do this work once and then save the entire VM to the disk for later use, creating a “Virtual Image”.   Then, when the user wanted – for example – another copy of their accounting system, they need merely direct the hypervisor to load the Virtual Image into memory and start it.   All of the system administration headache associated with creating a system to run a software application was eliminated. This feature has become very important to the growth and use of Cloud Computing.

1)      It abstracts the hardware so that all computer systems look the same to software running in a virtual machine, allowing the same software to be easily deployed on different systems, and making all systems look identical to the software.

2)      It allows creation of a number of virtual machines on each server, each of which may run completely unique software including different operating systems.

3)      It shares the hardware resources of the computer in such a way as to prevent any VM from impacting the performance of another (though of course they must each receive a fraction of the underlying hardware’s resources that may add up to no more than 100% of the available resources.)

4)      Because the VMs don’t “know” about each other since they cannot see each other (except on the network), each VM is secure from the other VMs and can serve users who do not want other users on the same computer to know they are there. VM security is generally accepted to be as good as the security between two different physical computers.

5)      The hypervisor allows creation, management, and deletion of virtual machines including setting the hardware resources assigned to each VM. This management can be initiated from a user interface or programmatically.

6)      Pre-built VMs, called Virtual Machine Images, can be easily loaded into the computer by the hypervisor, allowing almost instant provisioning of software applications.

Because of virtualization, it became possible to divide the resources of a physical server up among different paying customers according to their needs, while keeping the customers secure from each other’s software. This capability has allowed service providers to lease computer time very flexibly, only charging customers for what they requested or allocated, and changing that allocation on demand. In addition, customers could rapidly deploy their software to a virtualized server by copying pre-built images into it.

You may be asking yourself, what’s so new about the Internet? It’s been around for 20 years!   However, there has been a significant change in the last few years: large numbers of people have been connecting to it with broadband connections and cell phones, instead of the slow dial-up lines used in the 90s.  This growing population of broadband-enabled users are capable of doing much more with the internet and create a market for media, web applications, social networking, and web-delivered software services that has driven an explosion in the managed hosting and leased-server industry.   As more and more people create web-facing software, the demand for easy-to-use hosting and web servers has increased.   In addition, the demand for these web-based services as well as the relative ease of creating them and securing funding for startup companies that deliver them has reinforced the demand for managed hosting.

The combination of virtualization, which allowed managed service providers (hosting companies) to make dramatic cost reductions with shared hardware as well as greatly improving the ease of deploying customers’ software systems onto hardware, and the sudden increase in demand for managed hosting due to mass adoption of broadband internet, created an opportunity to sell self-service, pay-as-you-go, easily adjustable computing to millions more customers than ever before, at lower prices than ever before. This was the perfect storm that drove the creation of Cloud Computing.

Welcome to Cloud 101, a course on Cloud Computing. The intent of this course is to provide buyers of Cloud Computing with enough knowledge to purchase the appropriate Cloud Computing service for their business, and deploy their software on it. The class is going to be created on-the-fly in a blog format, so your feedback and comments will help to direct it towards your needs.

- a basic understanding of what is inside a computer system (CPU, memory, disks)

- a basic understanding of computer networking and the Internet

- a basic understanding of how software is developed and deployed

Feel free to skip any lesson in the series, or to jump around as your interests guide you. The first paragraph of each lesson will give you an idea of the topic material it covers.

Ask ten people what they think Cloud Computing is, and you’ll get ten different answers. This makes it exceedingly difficult to talk about it. It also makes it difficult for customers to compare, buy, and pay for it as well as for vendors to sell it. So before we get into the details, it makes sense to agree on at least a general definition of what Cloud Computing is.

Everyone talks about three basic characteristics of Cloud Computing:

1) Abstracted: In the past, if you wanted computing you’d buy or lease a computer. With the hardware (the computer) abstracted into service, you can now buy computing, which is what you wanted in the first place!

2) Scalable or Elastic: Unlike older ways of buying computing – where you had to buy a computer which gave you a fixed amount of computing power – with Cloud you can scale your computing usage up and down as your needs change and avoid having to overbuy to prepare for future needs. In the past, scaling meant changing out hardware, but now it’s as simple as clicking a button on a web page, or even just setting a policy whereby the amount of computing follows the demands for it.

3) Pay-as-you-go or utility billing: Since you can change how much computing you buy over time, it makes sense that it be sold to you like electricity, where you only pay for what you use

There are other characteristics that people often rely on and hold important about Cloud. These characteristics depend a lot on which cloud vendor you choose, and we will discuss them further during the class.

4) Uptime: Cloud is often assumed to be more reliable than simply leasing or buying a server or servers and putting them to work in your business, in part because experienced professionals are providing the service, and in part because the technology used to provide it is more sophisticated than simply one or more computers hooked together.

5) Security: Cloud is often assumed to be more secure or less secure than owning/leasing your own computer – depending on what the expectations are.

6) Cost: Cloud is often assumed to be cheaper than owning/leasing your own computer. Also, TCO (total cost of ownership) of a cloud-based IT strategy is assumed to be lower than having an IT department that owns its own computers.

People often categorize Cloud Computing into three levels of use model (or Cloud Computing services.) All of them satisfy the three characteristics listed above (#1-#3). The chart below shows how the three levels of cloud computing create the service the end user of the application software sees, and how it was provided before the days of Cloud Computing.

1) Infrastructure-as-a-service (IaaS): Cloud Computing replaces bare computer hardware. Customers manage their “cloud computers” from a system administration level, but don’t worry about setting up or maintaining any hardware. The user of IaaS is the person who would have bought the computer before Cloud Computing existed. Examples are companies such as a Amazon, ENKI, GoGrid.

2) Platform-as-a-service (PaaS): Cloud Computing replaces an execution environment for a computer language by providing a system ready to execute the user’s software. The user of PaaS is the programmer. Note that by installing the execution environment into an IaaS service, you produce your own PaaS service. Many choose to go that way because they have more control over the software environment. Examples are companies such as Engine Yard or Google.

3) Software-as-a-Service (SaaS): The user interacts directly with the Cloud-hosted software, and often pays for “seats” or “users” instead of computer time. Examples are NetSuite, SalesForce.com, Google Apps

These models are not exclusive, and can even be combined in many ways: for example, you may use SaaS from a company that purchases IaaS, or you may use software that your own administrators maintain but runs on an IaaS service from a cloud provider. How you use Cloud Computing really depends on where it can make a positive difference.in your value chain (how you do your work or create your product/service.)

This course will focus on Cloud Computing that is sold by the resource-hour, such as a CPU-hour or Memory-hour, Disk-usage-hour, etc. which is typically IaaS.

Another way to look at Cloud Computing is that it is a way to bring the benefits of the latest technological advances and economies of scale that a large corporate datacenter would have, to customers who cannot afford such a large expense. These technological advances include computer virtualization, multi-core servers, I/O virtualization, high capacity and performance storage, and automated management of the datacenter to provide failover and disaster recovery automatically. Large datacenters also enjoy economies of scale by eliminating middlemen and buying in bulk, savings which the average co-location customer simply cannot access.

Cloud Computing is a way to aggregate demand for data center services run under a highly automated virtualization management framework in such a way that users enjoy economies of scale and best-of-breed IT experiences without having to build the infrastructure for themselves.

A few weeks ago, I attended the Cloud Camp at Sun's Java 1 conference.  I learned something that really surprised me about Cloud Computing that I'd like to share with you.

I entered the conference excited about sharing some of my experiences as a Cloud vendor, and hoping that those experiences would benefit others at the Camp.   I'd already decided that I'd like to discuss "how to monetize your application in the cloud" as a topic, since so many of my customers ask me questions about this.

The Camp, being an un-conference, started out with the audience listing their questions and the moderator writing them down.  Then the moderator asked any members of the audience if they had expertise to address the questions.  I did, so I ended up on the panel answering questions.   While there were many questions about Cloud Computing, most seemed to focus on deployment issues such as cost and security.  Eventually we narrowed the questions needing more discussion down to a small subset, and broke up into groups to discuss them.  Since I'd suggested monetizing your app in the cloud as a topic, it got the votes and I ended up moderating a discussion session with that topic.

However, we barely were able to touch the topic.  Instead, almost every one of the 20 or so people in the group seemed to be more concerned about cost than revenue, even though the topic covered both.   Some were upset at the cost of EC2 which they had been using.  Others were confused about what "on-demand" means.  Others didn't understand how they could predict their costs.  And many just wanted to listen, frustrating my desire to have a stimulating conversation.  Almost everyone was confused about exactly what "Cloud" meant.   I ended up teaching a mini-class on how the choices they make end up resulting in the costs they have to pay for Cloud Computing, and what they can do to minimize cost.

This ended up pointing out to me that the Cloud industry has a problem.   If customers don't know what the product is, and they have high expectations of it as a result of lots of advertising and hype and excitment, they are bound for disappointment, and vendors are bound to have a hard time making them happy.  And the people at this Cloud Camp - unlike the geek-focus of prior Camps I've been to - were CTOs, Directors of IT, VPs of Technology and the like.  And they were confused.  If they didn't get it, how would COOs, CFOs, business owners, and entrepreneurs?

This resonated with me, since we founded ENKI on equality of power between us and our customers...  in other words, a win/win paradigm.  You can't have win/win if one side holds all the cards - or all the knowledge.  So for the industry to move forward, we will need to educate the customers.

To this end, I'm creating a "Cloud 101" class that I will be presenting here in this space, and eventually as live sessions at local events in the Bay Area, as well as video segments on this site.   The idea will be to present a vendor-neutral look at what Cloud Computing is, and how to use it successfully.

To continue with the un-conference format, I'd like to solicit suggestions from you on what you'd like to see in the class.  My current straw-man syllabus is:

- The on-demand computing model and the pay-as-you-go billing that results from it.

- Is Cloud Computing less expensive than alternatives?  How does it save you money?

- Reasons to choose Cloud aside from cost considerations.

- How to make money deploying your software to the Cloud?

- How should I write my software to run in the cloud?

- What are the differences between various Cloud technologies?

- Your topic here.