The Cloud Is Full

September 10, 2026 SupportTrenches Systems, Processes & Resilience 7 minutes

Nobody planned for the day elasticity ran out

For fifteen years, "the cloud is infinite" was the most reliable lie in enterprise IT. Not a malicious lie. A useful one. It let architects design for peak load without capacity planning, let CFOs treat compute as opex, and let me tell customers "just scale out" with a straight face.

Then customers in North America and the UK started filing tickets that read like they came from 2004. "Cannot provision." "Cannot reschedule." "Try another region." Across all three hyperscalers, in specific regions, at specific instance families. AWS reportedly told its own engineers to conserve compute "however they can" - internal teams waiting days for CPUs, because customer workloads come first. Which is the correct priority and also a sentence that should not exist in a business built on the promise of on-demand.

Ten years ago I would have laughed at anyone predicting this. The whole pitch was that Amazon, Google and Microsoft had so much spare iron that your workload was a rounding error. And they did. They just sold the rounding error to language models.

The number that shouldn't exist

Here is the part that should make every architect uncomfortable. Cast AI's 2026 telemetry across 23,000+ production clusters puts average GPU utilization at about 5%. On AKS, 2%. On EKS, 5%. On GKE, 6%.

Read that again, slowly, while remembering that half of GPU cloud providers reportedly sold out of H100 and H200 capacity, that reserved capacity is pre-sold six months out, and that AWS raised prices on ML Capacity Blocks - the first GPU price increase in over a decade of cloud GPU availability.

So we have a global shortage of the scarcest industrial input on earth, and the average buyer is using five percent of what they hoarded.

That is not a supply problem. That is a hoarding problem wearing a supply problem's coat. A 20-GPU H100 cluster idling at that rate burns roughly $168,000 a month producing nothing. Multiply across 23,000 clusters and you are looking at the largest voluntary transfer of shareholder value into warm air in the history of computing.

Take the ratio of what you pay to the actual compute-seconds of useful work inside it. When that ratio hits 20:1, the input isn't expensive. Your process is broken. Nobody in the shortage is short of GPUs. They are short of scheduling discipline. And because reserved capacity is a status symbol now - your competitor announced their cluster, so you need a cluster - the shortage is partly manufactured by people who want what other people visibly want.

The genuinely funny consequence: it is entirely possible that a meaningful chunk of the compute crunch strangling your deployment pipeline is caused by other companies' GPUs sitting idle at 2% because releasing them would look like retreat.

Why the free market can't just fix it

In a textbook market, prices rise, capacity follows, the curve resolves in a quarter or two. Amazon guided roughly $200 billion, and they explicitly credited memory prices for the bump. Google went to $600 billion collectively.

And they still say they won't have enough. Through 2027. With committed demand reaching into 2028.

Because money buys silicon, and silicon buys nothing without three things money is bad at purchasing quickly:

1. Fabs and HBM. High-bandwidth memory doesn't respond to a purchase order. It responds to multi-year fab construction, and the supply chain runs through geographies currently enjoying a level of geopolitical tension that no procurement team has a mitigation plan for. Memory, GPUs and even hard drives are constrained. PC and phone shipments are down because manufacturers can't get RAM. Your laptop is more expensive because a chatbot needed it more.
   
   
2. Land and consent. This one is the most instructive, and the most ignored by people who spend their lives in abstractions. Ask a town to host a data center and they do the math you'd do. What does the community get? A construction boom, then a permanent staff you could fit in a minibus - a few people watching dashboards, swapping a drive occasionally, and one very good HVAC technician. What does it cost? Grid load, water for cooling, and rate hikes. In Phoenix, the largest utility asked for a 14% residential increase and a 45% increase for large users, after a record demand month where roughly half of the growth came from data centers. Arizona lawmakers paused tax incentives for new data centers for three years. Around forty cities have signed a pact pushing for stricter standards.
   
   That is not NIMBYism ("Not In My Backyard"). That is a rational actor reading a term sheet where they hold all the downside and none of the upside. Skin in the game runs both directions. If you want a community to absorb your externalities, you have to give them something better than jobs that don't exist.
   
   
3. Electrons at 3 a.m. Here is the awkward part. We have more installed solar and wind than at any point in history, and the grid is still the binding constraint - because a training run has the demand profile of a smelter, not a household. Flat, enormous, indifferent to weather. Fifteen years of energy policy optimized for cheap average kilowatt-hours; AI needs guaranteed, dispatchable, always-on capacity next to a fiber trunk. Different product. The invoice for that mismatch is arriving now, itemized on residential electricity bills.

The double standard, briefly

I'm old enough to remember when Bitcoin's power consumption was a moral emergency. Op-eds, ESG memos, "each transaction boils a swimming pool" graphics of dubious arithmetic.

Bitcoin runs around 150-175 TWh a year. Data centers as a whole are around 415 TWh and climbing fast, with AI the fastest-growing slice. So AI is arguably competing in the same weight class, not orders of magnitude above - and much of what it produces is images of cats in armor and LinkedIn posts written by machines about the importance of authenticity.

Nobody is running protests about that. Not because the numbers changed. Because the narrative did. Energy consumption was never the objection. It was rhetoric dressed as arithmetic, and once the consumer moved from "speculators" to "innovation," the arithmetic quietly stopped mattering.

I don't say this to defend either. I say it because if you can't tell when you're being handed rhetoric instead of a measurement, you will lose every budget fight you enter against someone who can.

What this means for you on Monday

👉 Your DR plan has a fiction in it. "We fail over to region B" assumes region B has room. Test it. Actually try to provision your full failover footprint. Some of you will discover your business continuity plan is a poem.

👉 Capacity planning is a job again. Hire or grow someone who can forecast. That skill was declared obsolete around 2014 and the people who kept it are about to be very well paid. Meanwhile the power horse 🐎 in your infra team who has been complaining about idle reservations for two years is currently sitting there being right, underpaid, and unread.

👉 Measure utilization before you buy anything. If you are at industry average, you can 5x your effective AI capacity without touching the market. Best part is no part: the cheapest GPU is the one you already have, awake.

👉 Reserve pessimistically, engineer optimistically. Checkpoint frequently. Make jobs restartable and preemption-tolerant. Fall back gracefully. Treat compute like a spot commodity, because that is what it became while you were writing your cloud strategy deck.

👉 Assume shortage is the new normal through 2027 at minimum. Both the hyperscalers and JP Morgan's supply-demand gap estimate of 15-30% agree. Architect for scarcity, and if abundance returns, enjoy the surprise.

We spent a decade teaching a generation of engineers that infrastructure was weather - always there, never their problem. The bill for that abstraction arrived as a provisioning error, and it turns out the cloud was never infinite. It was just, until recently, underbooked.

👇 Have you actually tried to provision your full failover capacity in your secondary region lately - or is that still a diagram?

This article is also available in German.

HardwareTechnologyCloudUSAAIFuture