Overview - How do Cloud Providers Pack 'em In

When I speak with potential and current customers of cloud computing, the question I'm asked most often is "how much will it cost"?   That depends primarily on three things: how much of "it" the customer wants, how providing it affects the provider's costs, and of course how the provider does its pricing, which we will cover in another lesson.  The fundamental principle behind all cloud computing is shared infrastructure (computers, network, and storage) to provide computing services to many users.  As we saw in Lesson 2, virtualization is the key to accomplishing this by allowing many customer applications to share a powerful server without being affected by each others' resource usage and by providing security between customers' applications.  But virtualization alone doesn't determine how the provider allocates their infrastructure to each customer.  Instead, what determines the provider's costs is largely their packing strategy.  This lesson will explain why packing strategies are important in determining what a customer pays for cloud computing and how flexible the cloud computing offering is, as well as discussing the most common packing strategies and their advantages and disadvantages.  Finally, the lesson will explain why cloud vendors can never exactly explain the relationship between cloud computing performance and cost.

In this lesson and in the future, we will be talking about Total Cost of Deployment (TCD), which is the actual cost of putting your application into the cloud, versus what a vendor charges or claims to charge.  TCD is ultimately what customers are interested in, but unfortunately, they usually don't have enough information to determine it, being stuck with vendors' per-hour or per-instance charges instead, which may vary greatly from TCD.  Calculating TCD is so complex, that this single obstacle represents a significant threat to the future of cloud computing as  it exists today, since cloud customers are led to believe that by outsourcing the infrastructure, they are also outsourcing the complexity of calculating what it costs.  Many customers may choose to either be less focused on cost and elect to trust that their vendor is using the underlying cloud technologies to give them a fair deal, or they may seek out fixed-cost packages that include a range of resources without requiring an understanding of the costing details.  If this matches the approach you wish to use to relate to your cloud vendor, you can skip the rest of this lesson.

Packing Strategies

To understand why a packing strategy is important, let's start with a simple analogy: packing oranges into a crate.  The number of oranges you can get into a crate depends on how big the crate is, how big the oranges are, and how you pack them in, which is the packing strategy.  If you just throw the oranges in, you they will jumble up and leave lots of free space.  Instead, packing houses place the oranges into the crate in an optimal way such that every empty space is filled.  The obvious parallel to cloud computing is that the crate is a provider's server, and the oranges are customer's software applications.  However, cloud customers' needs aren't exactly like oranges: they vary widely in "size", with some software applications requiring more CPU or memory than others.  Using the regular packing strategy that works for oranges to pack many customers' applications into a single server won't satisfy most customers who want cloud computing to meet their varied needs.

Now that you understand why the packing strategy is important, let's look at an example that more closely matches how cloud providers allocate server resources to customers: placing families in houses.  To satisfy the needs of a family well, a house must have the right number of bedrooms and bathrooms.  Too few of either and people can suffer.  Too many, and they pay too much rent or mortgage.  Here, the bedrooms and bathrooms correspond to CPU power and memory usage in the cloud computing world, and the housing development corresponds to a cloud server.   If we measure the housing efficiency - the efficiency of the packing strategy - of a neighborhood by how well its houses satisfy its homeowners on space and payments, the best efficiency would occur if each family had the number of bedrooms and bathrooms that it needed.   But families come in all different sizes, and to make the problem of efficient packing even harder, they tend to change in size.  If families were forced to live in just a few house designs (sounds like a country I wouldn't want to live in) then pretty much everyone would either be paying too much for their housing or struggling with not enough space and resources.   Instead, we have come up with ways to improve the packing strategy for housing: families can move to another house if theirs no longer efficiently meets their needs, and they can remodel if they want to stay in the same location, or even rent out extra rooms to downsize the space they're paying for.  As we shall see, each of these strategies for optimal packing corresponds to techniques that cloud providers use.

There are three packing strategies that cloud providers use today:

1)      Fixed Instance Size Allocation, in which customers are offered a choice of virtual computer "instances" with pre-allocated quantities of CPU and memory.

2)      Quantized Resource Allocation, in which customers can build their own virtual computer by setting the resource allocations from a limited choice of quantities.

3)      Variable Resource Allocation, in which customers can choose whatever allocation they please.

Each strategy has advantages and disadvantages in terms of resource costs and limitations on flexibility and manageability that are imposed on the customer, though each is in keeping with the basic premise of cloud computing that each customer is guaranteed the resources they ask for.   In the sections of this lesson that follow, we will discuss each one and how it affects what the customer pays.

There is also a fourth packing strategy in which customers are not guaranteed the resources they ask for.  In this strategy, the server's resources are split up using pseudo-virtualization that divides its resources among all the customers' instances depending only on how many instances are loaded on the server.  This can result in very poor performance as well as potentially betraying or cheating the customer, depending on what was promised to them.  On the other hand, this allows vendors to offer services at very low cost, like coach class on a packed airplane.  Many "hosting" providers offer this service under the banner of cloud computing.  But this is not truly cloud computing, because one of the requirements - hardware abstraction, which hides the details of the underlying hardware's capabilities - is not met.   However, it may meet the needs of many customers who need very little resources, such as very small businesses or pre-launch web entrepreneurs.