AI boom could lead to a new GPU shortage, but Nvidia says the current problem is just...

[Daniel Sims]

Why it matters: The crypto boom caused a severe spike in PC graphics card prices throughout 2020 and 2021, and prices still aren't where many users would like them. The current AI craze could have a similar effect as some companies report shortages, but Nvidia says the supply chain is more complex than many assume.

Recent statements from AI companies indicate that the AI boom is increasing the demand for GPUs to the point where some report shortages. Nvidia claims its supply chains haven't totally failed to meet demand but are encountering minor snags, somewhat allaying fears.

Microsoft, OpenAI, and other companies mention steps they've had to take to mitigate shortages in Nvidia H100 and A100 enterprise GPUs, which are used for AI workloads. Microsoft is rate-limiting employees and the CEO of Quora says the hardware shortage is hiding the real potential of AI applications. Elon Musk joked that enterprise GPUs are currently harder to acquire than drugs.

AMD (unlike Qualcomm) is a joy to buy chips from. 7.38 PFLOPS of compute was delivered today, with exaflops more to come.

Nice to see them more active on the ROCm GitHub, we are working on the layer above @__tinygrad__ pic.twitter.com/5yE11TVA5F

– George Hotz �' (@realGeorgeHotz) August 1, 2023

Nvidia explained that oftentimes, the manufacturing hangup with GPUs doesn't involve the main processing unit but rather some other component of the add-in board. In the current case, the problem is packaging, which the company currently handles with TSMC's 3D stacking chip-on-wafer-on-substrate (CoWoS) technology. Nvidia promises that supply will be much stronger in the second half of 2023.

It's unclear how much the situation could spill over into the consumer GPU market, but the AI boom has already caused Nvidia to divert some production resources from GeForce RTX 4090 graphics cards toward H100s. While comments from AI companies imply that shifting attention toward AMD hardware would take time, a post from prolific hacker George Hotz indicates that his company, comma, is purchasing Radeon 7900 XTX cards for that purpose.

The Green Team is currently dominant in AI, but TechSpot's analysis suggests AMD can certainly make significant moves in the sector. The company's primary advantage in AI is its lead in Windows PC CPUs that incorporate AI blocs.

Earlier this year, groups that had previously used GPUs for mining began attempts to pivot their infrastructure toward AI workloads to recoup investments after the crypto winter. Although the shift may prove fruitful for some, it will likely be more challenging than mining due to competition from major players like Microsoft, Amazon Web Services, and OpenAI.

Permalink to story.

https://www.techspot.com/news/99682-ai-boom-could-lead-new-gpu-shortage-but.html

 

[gigantor21]

Not very broken up about this generation of cards going out of stock.

 

[yRaz]

Not very broken up about this generation of cards going out of stock.

why? This is will just reinforce the current pricing and encourage scalping again.

 

[loki1944]

"AI is BS. And for the record, this naysayer taught the Columbia University graduate-level “Artificial Intelligence” course, as well as other related courses there.

AI is nothing but a brand. A powerful brand, but an empty promise. The concept of “intelligence” is entirely subjective and intrinsically human. Those who espouse the limitless wonders of AI and warn of its dangers – including the likes of Bill Gates and Elon Musk – all make the same false presumption: that intelligence is a one-dimensional spectrum and that technological advancements propel us along that spectrum, down a path that leads toward human-level capabilities. Nuh uh. The advancements only happen with labeled data. We are advancing quickly, but in a different direction and only across a very particular, restricted microcosm of capabilities.

The term artificial intelligence has no place in science or engineering. “AI” is valid only for philosophy and science fiction – and, by the way, I totally love the exploration of AI in those areas."

https://bigthink.com/the-future/why-a-I-is-a-big-fat-lie/

 

[Mr Majestyk]

In further news, Nvidia has basically stopped all production of Lovelace. They would rather squeeze supply than drop prices and basically don't give a fcuk. The pc GPU market is now peanuts to them. They are even cutting supply of the last gen AI cards like A100 and forcing companies onto the 3-5x dearer H100 even if they don't need it.

In even worse news plenty of people are reporting AMD has cancelled RDNA4 N41 and N42. it now appears next gen will be only low end crap and high performance won't be available again until RDNA5 maybe in 2026. 7900 series will have to carry the flag for 3 years+. And let me say I do not believe it should be taken for granted RDNA5 will even come to light. People better be hoping Intel can make something out of Battlemage next year, because AMD it also appears, doesn't give a toss either about pc gpu's.

 

[takaozo]

In other words the used market hit them so hard along with customers refusing to buy new at megascalpers prices that they slow down the chip production. After the AI bubble will pop I realy dont know if Automotive and consoles will keep them afloat.

 

[Bobbydpue]

In further news, Nvidia has basically stopped all production of Lovelace. They would rather squeeze supply than drop prices and basically don't give a fcuk. The pc GPU market is now peanuts to them. They are even cutting supply of the last gen AI cards like A100 and forcing companies onto the 3-5x dearer H100 even if they don't need it.

In even worse news plenty of people are reporting AMD has cancelled RDNA4 N41 and N42. it now appears next gen will be only low end crap and high performance won't be available again until RDNA5 maybe in 2026. 7900 series will have to carry the flag for 3 years+. And let me say I do not believe it should be taken for granted RDNA5 will even come to light. People better be hoping Intel can make something out of Battlemage next year, because AMD it also appears, doesn't give a toss either about pc gpu's.

Lets not get upset about speculation or situations that, for now, are fictional.

 

[Neatfeatguy]

"...the AI boom has already caused Nvidia to divert some production resources from GeForce RTX 4090 graphics cards toward H100s."

That would be because very few people are buying up the 4090s after the initial release of them. The first 6 weeks or so it was hard to find a 4090, after that it has become impossible to find a place that doesn't have stock of the 4090 because everyone that wanted a $1600+ GPU got one.

Just looking at my local Micro Center, they're sitting on 10 various models of the 4090. Out of those models they show a total of 54 in stock. The lowest priced model (MSI Gaming X Trio) is at $1700 and the highest priced model (ASUS ROG Strix LC Overclocked Liquid Cooled) is at $2200.

Awful pricing, if you ask me, which is why they're just not moving anymore. To have Nvidia shift "some production" away from the 4090 isn't going to impact anyone since people really aren't buying them.

 

[ThrakazogZ]

I'm sure "shortages" will occur from now on, whenever companies feel the need for higher profit margins.

 

[Mr Majestyk]

Lets not get upset about speculation or situations that, for now, are fictional.

The reports are coming from inside AMD.