On being the cheapest cloud

 September 30, 2024

Recently, I heard a pitch from a public cloud company. Among other characteristics, a key aspect they stressed is that they are the cheapest cloud. This aspect struck me. Not because I believe it is or is not, but because I’ve heard many companies pitch themselves as the cheapest cloud over the years. I asked the CTO if they were foreseeing consistent and planned cuts in the pricing every year or so. The CTO’s answer was very sensible but negative on the specific point. Later the same day, I was thinking more about my interaction with that CTO, and it became clear to me why pitching to be the cheapest cloud is not a good idea.

Before we focus on the specific point, we need to remember Moore’s Law: “The complexity for minimum component costs has increased at a rate of roughly a factor of two per year.”. Although Moore made this prediction in 1965, it still holds with minimal correction coefficients. Even more interestingly, Moore himself published a paper in 1995 where he highlighted that the same exponential behavior also works for the majority of other metrics related to electronics, including the cost.

Measuring the cost of computing or storage over many generations is a very complex matter because the results can greatly change depending on what you use as a comparison. In our analysis, I will refer to the per-core cost for CPUs considering the latest released Xeon processor with the following characteristics:

• highest core count
• highest frequency
• excluding OEMs-only processors
• excluding processors designed for multi-processor setups
• excluding processors having higher prices due to the presence of special accelerators

Although the TDP is critical to understanding the Total Cost, I’ll ignore and focus exclusively on the acquiring cost. I expect that an analysis of Total Cost will report very similar results. In addition, 3'000$ will be added per processor to account for the other components required to make a processor useful (motherboard, case, power supply, etc). I will use Wikipedia as the source for the MSRP prices at the release date. More specifically, I will consider the following processors:

• 2003 - Xeon MP 3.0, 1 core, 3GHz, 85W TDP, 3692$ => 6692$/core
• 2008 - Xeon E3120, 2 cores, 3.17GHz, 65W TDP, 188$ => 1594$/core
• 2013 - Xeon E5-1680 v2, 8 cores, 3GHz, 130W TDP, 1723$ => 590$/core
• 2018 - Xeon D-2191, 18 cores, 1.6GHz base and 2.2/3.0GHz with Turbo, 86W TDP, 2406$ => 300$/core
• 2023 - Xeon 8471N, 52 cores, 1.8GHz base and 2.8/3.6GHz with Turbo, 300W TDP, 5171$ => 157$/core

As for storage, I will refer to the per TB cost, regardless of IOPS or bandwidth improvements and not accounting for data replication, taking the data from Our World in Data. For the years until 2012, the SSD cost will be the “Flash” cost.

We can, therefore, create the following table:

Looking at this table, it becomes clear that the prices of compute cores and storage decrease very quickly. The CPU per core cost roughly halves every five years, which means it goes down roughly 12% annually. The HDD Storage GB cost shows similar behaviors. The cost decrease for SSD Storage is even more sensible, with a roughly 25% per year decline.

Looking at those prices, it becomes apparent that if a new cloud provider is founded in year X and sells the cloud resources at the minimum possible price to achieve perfect parity between revenue and direct costs, in year X+5, they will make 50% of profit on direct costs if they do not change the pricing they apply to their customers. This is not exactly the reality because all other components (rack space, energy, etc.) of the direct costs do not decrease at the same rate. Also the company in question will have indirect costs to be accounted for. Still, the profits will increase significantly. As a consequence, in year X+5, a new cloud provider can be created that will cost much less than the one launched in year X and still not loose money.

Considering all of this, either the cloud providers lower their prices sensibly every few years or they allow new (cheaper) cloud providers to enter the market. At this point, an obvious question arises: why don’t cloud providers reduce pricing then (or do they do it very rarely)? Let’s assume that a cloud provider sells $1M/year of computing and $1M/year of SSD storage and has costs for $1.5M/year. Thus, they have a total revenue of $2M and a profit of $0.5M/year (25% of margin). Assuming that the costs and customers do not change, if this cloud provider reduces the compute price by 10% and the SSD storage price by 20% (since this will apply to all new and current customers), they will end up invoicing 0.9M$/year of computing and 0.8M$/year of SSD storage, totaling 1.7M$/year of revenue and 0.2M$/year of profit (12% margin). As you can see, both the profit and the margin are more than halved. Therefore, such a price reduction is tough to justify to owners and investors, which means that very rarely cloud providers are in a position where they are allowed to cut prices.

Another interesting consequence of these calculations and considerations is that if you compare a specific cloud provider to building a private datacenter, the cloud provider becomes less convenient every year, at least from a pure cost perspective.