"Dark GPUs" at Amazon show the limits of the AI buildout: chips are easy to buy, power is not

[Alfonso Maruccia]

Dark AI Clouds: Like any other hyperscaler willing to sacrifice everything to turn AI into a game-changing opportunity, Amazon is pulling some bold stunts with both its finances and its hardware. A new report says the company is buying more GPUs than it has the power to run.

Amazon is allegedly taking in billions of dollars' worth of Nvidia racks every month, even though some sites aren't ready for them and there is no electricity to feed the AI accelerators just yet. According to internal emails seen by Business Insider, Amazon is willing to tie up billions in cash upfront as long as it can secure new hardware capacity early.

The messages describe Amazon's Early Rack Receipt process, through which AWS expected to take around $5 billion worth of Nvidia GB300 server racks in August alone, with roughly $3 billion more expected every month for the foreseeable future. Amazon is getting servers "built for the age of AI reasoning" before the final deployment phase.

Some US-based AWS facilities don't have enough power to install and turn the servers on, which means the costly hardware is being stored in "intermediary" warehouses until the final data center site is completed. In the emails, one Amazon team lamented that there was no clear forecast for actually installing all the GPUs the company was acquiring each month.

Amazon's lack of energy capacity is part of a growing phenomenon known as "dark GPUs." Data center and cloud giants are piling up all the hardware they can, but much of it is essentially gathering dust on shelves while the rest of the IT industry is left with the scraps.

The dark GPU issue is also adding financial pressure to data center ventures. Amazon's trailing free cash flow turned negative in the second quarter for the first time since 2023, and the company expects it to stay that way for a while, with capital expenditures set to reach around $220 billion this year.

Amazon is spending more than any other hyperscaler to build new AI capabilities and data center capacity, and is changing its usual business practices to do it. Before the AI craze, AWS only took ownership of racks once they were finished and delivered to a data center. Now, the company Jeff Bezos founded is willing to spend billions upfront to secure more AI racks earlier in the supply chain.

An Amazon spokesperson defended the approach, calling the practice of storing finished racks in a warehouse "best business practice." The company expects significant financial returns from this massive AI infrastructure build-up eventually, because AI is a once-in-a-lifetime opportunity no IT company can afford to miss. AWS has been through something similar before, Amazon CEO Andy Jassy said.

Permalink to story:

"Dark GPUs" at Amazon show the limits of the AI buildout: chips are easy to buy, power is not

 

[yRaz]

Everyone is doing this.

Even if there was a magic switch you could turn to power ever datacenter on with, these things take time to build and there simply aren't enough workers to build all the datacenters on the timeline.

It's not just a "not in my back yard" type thing with data centers, the capacity doesn't exist to build all the buildings they need. Everything from concrete plants, iron, copper wire, electricians, people who actually assemble the servers.

It's easy for them to point to politics and permits but they knew long before they bought the GPUs that many of these buildings could take several years to complete. And many of these companies have a spoiled brat I want it now additudes so the built parts of the structure anyway. Now it's everyone else's fault when the skilled labor to put this Lego set together isn't around.

The knew this before buying all the GPUs, the constant walls they ran into it confirmed what they already knew was true and they skill keep buying billions of GPUs every month

GPUs are the new tulip mania

 

[noel24]

Jensen: "Gold is there, I'm telling You, You just have to buy my shovel"
It's a bubble, no matter what They say. If You have to buy equipment years before You'll be able to deploy It, It's not an organic growth, It's an overinvestment. It's a bubble.

 

[James321]

"AWS is reportedly buying billions in Nvidia GB300 racks every month and storing them in warehouses"

Why buy it then ??? If you can't use it. They just artificially inflating the price.

Why !!! Not wait and get the power sorted and then slowly build up the AI Data centers?

So they wasting money buying GPU's at inflated prices that they can't use. The reason why the memory prices is out of whack as well.

Well bloody done, just shows being rich doesn't mean you are clever or smart with money...

 

[yRaz]

"AWS is reportedly buying billions in Nvidia GB300 racks every month and storing them in warehouses"

Why buy it then ??? If you can't use it. They just artificially inflating the price.

There are 2 major and equally rediculous reasons. 1)Nvidia calls you and says "we have GPUs to sell, buy them now or you risk being able to do business with us in the future. 2) They buy them simply so their competitors DON'T have them.

The memory companies are duing the same thing

 

[takaozo]

I used to work as a Sysadmin for many years, this is common practice. Buy now deploy later.
From sending the order to the deployed status and depending on the project 1 to 6 months. That's when you already have the datacenter. I have seen cases where was more than 12 months because of some license that must be migrated or software not ready. Aka development delays.

But buying now and store it until the datacenter is build is on another level. If more than 2-3 years pass the hardware is old already.

 

[gdavid65]

Unfortunately those that may actually want use such products day to day (or well at least the memory capacity), right now get ****ed over... while these racks and components gather dust.

 

[Mtx26]

Then build power.

And build power for the nation at the same time.

SMR nukes would be ideal

 

[GeoffreyA]

The universe's energy laws are wonderful.

 

[yRaz]

Unfortunately those that may actually want use such products day to day (or well at least the memory capacity), right now get ****ed over... while these racks and components gather dust.

The homelab scene is gonna be silly amounts of fun in about 4 years. For running services and stuff, a new miniPC connectd to your network is usually the best way to go, but having a rack full of of old hardware as a toy with hundreds of cores and stupid amounts of ram can be a lot of fun. I volunteered at an e-waste recycler in college and saw all kinds of fun stuff and they would even sell lots of stuff to me at scrap metal prices. I have 8 xeon e-2236 systems (4, 2 socket 2u racks)I got for pennies on the dollar. I wouldn't want to run anything serious on them 24/7, but for what I paid for them, it's absolutely a blast turning them all on a few times a year if I find some workload that seems fun. I was considering replacing all of them ARM chips with more cores and memory, but then this stupid market happened. I know this might not sound that exciting today, but having 48 cores and a terabyte of memory to play around with is a lot of fun and I got all that for less than the cost of a single threadripper CPU.

 

[Inthenstus]

The homelab scene is gonna be silly amounts of fun in about 4 years. For running services and stuff, a new miniPC connectd to your network is usually the best way to go, but having a rack full of of old hardware as a toy with hundreds of cores and stupid amounts of ram can be a lot of fun. I volunteered at an e-waste recycler in college and saw all kinds of fun stuff and they would even sell lots of stuff to me at scrap metal prices. I have 8 xeon e-2236 systems (4, 2 socket 2u racks)I got for pennies on the dollar. I wouldn't want to run anything serious on them 24/7, but for what I paid for them, it's absolutely a blast turning them all on a few times a year if I find some workload that seems fun. I was considering replacing all of them ARM chips with more cores and memory, but then this stupid market happened. I know this might not sound that exciting today, but having 48 cores and a terabyte of memory to play around with is a lot of fun and I got all that for less than the cost of a single threadripper CPU.

Sell the RAM now lol

 

[yRaz]

Sell the RAM now lol

It's DDR3 ECC and the cost to replace it wouldn't even come close to what I would need to replace it.

 

[Kirby1]

I think the AI bubble theory is mostly copium.

For one, these GPUs only loose value when something better can replace them, and when that happens I'd expect the roll out to be slow, so big companies won't be able to upgrade quickly meaning the older hardware will be viable for longer.

Second, only the companies in the middle of the stack are at risk. If they go under, they'll just be absorbed by the bigger guys. If OpenAI goes under, Microsoft aquires them. If Oracle goes under, AMD probably would aquire them. What's bad for one company is good for another.

TBH, I don't even think there will be any significant corrections outside of interest rates going up again, and even then I think it will be small.

I'd like to see the bubble burst as much as anyone here, but I don't think it's actually going to happen.

 

[LordVile95]

I think the AI bubble theory is mostly copium.

For one, these GPUs only loose value when something better can replace them, and when that happens I'd expect the roll out to be slow, so big companies won't be able to upgrade quickly meaning the older hardware will be viable for longer.

Second, only the companies in the middle of the stack are at risk. If they go under, they'll just be absorbed by the bigger guys. If OpenAI goes under, Microsoft aquires them. If Oracle goes under, AMD probably would aquire them. What's bad for one company is good for another.

TBH, I don't even think there will be any significant corrections outside of interest rates going up again, and even then I think it will be small.

I'd like to see the bubble burst as much as anyone here, but I don't think it's actually going to happen.

They’ve already lost value because everyone already has more stock than they can deploy and the only people buying these GPUs are the hyperscalers who already have their own supply.

I don’t think you understand how this works. There is no profitable product at the end of this that will make the trillions required to pay back the loans and investment this has needed. The only reason AI services have been adopted is because it’s being sold below cost. To make a profit they’re all going to have to up prices at which point they’re going to be dropped like a bad habit.

The hope was that AI capability would scale with hardware and that just hasn’t happened. Is ChatGPT that much better now than it was 18 months ago? Googles AI can’t even get the colour of a crisp bag right when I asked it “best Doritos flavour UK”

 

[Kirby1]

They’ve already lost value because everyone already has more stock than they can deploy and the only people buying these GPUs are the hyperscalers who already have their own supply.

I don’t think you understand how this works. There is no profitable product at the end of this that will make the trillions required to pay back the loans and investment this has needed. The only reason AI services have been adopted is because it’s being sold below cost. To make a profit they’re all going to have to up prices at which point they’re going to be dropped like a bad habit.

The hope was that AI capability would scale with hardware and that just hasn’t happened. Is ChatGPT that much better now than it was 18 months ago? Googles AI can’t even get the colour of a crisp bag right when I asked it “best Doritos flavour UK”

Copium.

 

[yRaz]

Copium.

What are people coping from?

 

[Kirby1]

What are people coping from?

I think that's fairly obvious. People want there to be a crash since prices for hardware are so messed up, and virtually no one has any good will towards these companies at this point. I'm sure we could list a lot more reasons as well.

People want it to come true a lot more than it's actually likely to.

Just look at the 5090, GPUs are only getting more valuable. The main thing that makes people think a bubble will happen is hardware waiting to be brough online. That hardware is very unlikely to lose significant value before it can be used with the way things are going. Sure, if markets were normal and companies were stockpiling GPUs anyway, that would be a big risk; but that isn't reality today. We have people who bought 5090s and 5080s selling them for a big profit and switching to AMD instead now, and these big companies are not paying the retail prices that consumers are.

 

[JaredTheDragon]

Copium.

Doltium.

 

[Kirby1]

Doltium.

Sorry to be objective instead of simply thinking what I want to happen is likely to happen.

Disagree all you want, but do it with reason or don't bother me with your whining.

 

[LordVile95]

I think that's fairly obvious. People want there to be a crash since prices for hardware are so messed up, and virtually no one has any good will towards these companies at this point. I'm sure we could list a lot more reasons as well.

People want it to come true a lot more than it's actually likely to.

Just look at the 5090, GPUs are only getting more valuable. The main thing that makes people think a bubble will happen is hardware waiting to be brough online. That hardware is very unlikely to lose significant value before it can be used with the way things are going. Sure, if markets were normal and companies were stockpiling GPUs anyway, that would be a big risk; but that isn't reality today. We have people who bought 5090s and 5080s selling them for a big profit and switching to AMD instead now, and these big companies are not paying the retail prices that consumers are.

The 5090 is getting more valuable due to the fact they’re out of production and are the best card for local LLM workloads. Same with all Nvidia GPUs at the price point. They’re not making many which pumps up the price.

 

[Tom Yum]

For one, these GPUs only loose value when something better can replace them, and when that happens I'd expect the roll out to be slow, so big companies won't be able to upgrade quickly meaning the older hardware will be viable for longer.

That is not how the market works though. These GPUs will lose value when supply exceeds demand, irrespective of whether there are newer models or not. A GPU that has been sitting in a warehouse for 3 years will sell for less than a brand-new one straight off the Nvidia production line, because it has less/no warranty. Secondly, while Nvidia would probably love to just keep cranking out B200s and H200s and jack the price each year, AMD and the hyperscalers themselves are actively trying to win marketshare/reduce dependence by producing better hardware, which will force Nvidia to keep releasing better products with better performance per watt and raw performance, depressing the value of older stock.

Second, only the companies in the middle of the stack are at risk. If they go under, they'll just be absorbed by the bigger guys. If OpenAI goes under, Microsoft aquires them. If Oracle goes under, AMD probably would aquire them. What's bad for one company is good for another.

This I agree with as the likely outcome. But even the top end hyperscalers and chip makers will experience a big hit to share prices, because their current multiples are based on AI revenue projections and TAM that are many times higher than anything reality can provide. When your revenue projections require every single person on the planet to spend over $US3500 a year on AI as Anthropic's currently does (remember, most of the world earns less than $3500 a year), it isn't copium to point out the unlikeliness of that reality being realised. And when current share prices are based on that projection, the reevaluation will be brutal when it happens. The MS's, Nvidia's, Google's, Meta's of the world will survive, but 401k's will be badly bruised.