HBM4 could double memory bandwidth to 2048-bit

[AlphaX]

In context: The first iteration of high-bandwidth memory (HBM) was somewhat limited, only allowing speeds of up to 128 GB/s per stack. However, there was one major caveat: graphics cards that used HBM1 had a cap of 4 GB of memory due to physical limitations. However, over time HBM manufacturers such as SK Hynix and Samsung have improved upon HBM's shortcomings.

HBM2 doubled potential speeds to 256 GB/s per stack and maximum capacity to 8 GB. In 2018, HBM2 received a minor update called HBM2E, which further increased capacity limits to 24 GB and brought another speed increase, eventually hitting 460 GB/s per chip at its peak.

When HBM3 rolled out, the speed doubled again, allowing for a maximum of 819 GB/s per stack. Even more impressive, capacities increased nearly threefold, from 24 GB to 64 GB. Like HBM2E, HBM3 saw another mid-life upgrade, HBM3E, which increased the theoretical speeds up to 1.2 TB/s per stack.

Along the way, HBM slowly got replaced in consumer-grade graphics cards by more affordable GDDR memory. High-bandwidth memory became a standard in data centers, with manufacturers of workplace-focused cards opting to use the much faster interface.

Throughout the various updates and improvements, HBM retained the same 1024-bit (per stack) interface in all its iterations. According to a report out of Korea, this may finally change when HBM4 reaches the market. If the claims prove true, the memory interface will double from 1024-bit to 2048-bit.

Jumping to a 2048-bit interface could theoretically double transfer speeds again. Unfortunately, memory manufacturers might be unable to maintain the same transfer rates with HBM4 compared to HBM3E. However, a higher memory interface would allow manufacturers to use fewer stacks in a card.

For instance, Nvidia's flagship H100 AI card currently uses six 1024-bit known good stacked dies, which allows for a 6144-bit interface. If the memory interface doubled to 2048-bit, Nvidia could theoretically halve the number of dies to three and receive the same performance. Of course, it is unclear which path manufacturers will take, as HBM4 is almost certainly years away from mass production.

Both SK Hynix and Samsung believe they will be able to achieve a "100% yield" with HBM4 when they begin to manufacture it. Only time will tell if the reports hold water, so take the news with a grain of salt.

Permalink to story.

https://www.techspot.com/news/100174-hbm4-could-finally-double-memory-bandwidth-2048-bit.html

 

[noel24]

Good news: even better AI Accelerators. Bad news? Your future consumer video card will be Intel or Chinese, as both nvidia, and soon AMD slowly already forgeting how to make one.

 

[yRaz]

Good news: even better AI Accelerators. Bad news? Your future consumer video card will be Intel or Chinese, as both nvidia, and soon AMD slowly already forgeting how to make one.

I don't know if it's time to start worrying about AMD just yet. So many of their GPU sales come from consoles, not gaming desktops, that I think they will always be making gaming GPUs. Not saying it wont happen, but as long Microsoft and Sony keep buying AMD GPUs they'll keep making them.

Not saying it's impossible but I do find it unlikely in the next 5 years. People seem to forget that AMD took out massive loans to develop RDNA and Ryzen CPUs. I don't know if they even broken even from the 6000 series because of how long they had to sit on shelves and how many price cuts they received. My point with that being is that they have to pay off the banks first before they can start speculating on AI workstation GPUs. People are buying their gaming GPUs for AI work to save money, though.

I have nothing to confirm what I'm about to say, but I've thought for awhile that the massive memory they put on the 7900xt/xtx was to make it more of a dual purpose card that can game and do AI. I know several college students who picked up a 4090 to use for AI research.

 

[noel24]

I don't know if it's time to start worrying about AMD just yet. So many of their GPU sales come from consoles, not gaming desktops, that I think they will always be making gaming GPUs. Not saying it wont happen, but as long Microsoft and Sony keep buying AMD GPUs they'll keep making them.

Not saying it's impossible but I do find it unlikely in the next 5 years. People seem to forget that AMD took out massive loans to develop RDNA and Ryzen CPUs. I don't know if they even broken even from the 6000 series because of how long they had to sit on shelves and how many price cuts they received. My point with that being is that they have to pay off the banks first before they can start speculating on AI workstation GPUs. People are buying their gaming GPUs for AI work to save money, though.

I have nothing to confirm what I'm about to say, but I've thought for awhile that the massive memory they put on the 7900xt/xtx was to make it more of a dual purpose card that can game and do AI. I know several college students who picked up a 4090 to use for AI research.

And some leaks suggest that most of AMD's limited TSMC silicon output will go to the server space production.
https://www.techspot.com/news/100112-leaks-suggest-there-wont-radeon-rx-8800-or.html
AI is just a buzz right now, and only thing that stops Chinese mobile-derived GPUs from conquering the World are drivers.
Allegedly (according to Moore's Law is Dead YT channel, in My opinion relatively reputable source) AMD not only didn't forced Bethesda to sabotage Starfield, but simply nVidia didn't put any resources into the game ready driver. AMD had no choice as They chosen It as their supported game (still with lots of pain), but giving as AMD is just step behind nVidia in Their ill choices, I can imagine that console support is the only thing that will make AMD drivers any half-decent in the near future, until AI craze will go away.
Start hoping Chinese will invest in game drivers developement. Americans just won't, as It doesn't bring as much money as AI.
And I'm Polish, so no big fan of dictatorships in the East.

 

[Theinsanegamer]

Good news: even better AI Accelerators. Bad news? Your future consumer video card will be Intel or Chinese, as both nvidia, and soon AMD slowly already forgeting how to make one.

Why does everyone believe this? Nvidia isnt going to abandon a multi billion dollar industry (even in a bad year, the geforce division brings in over 4 BILLION) and just hand it to their rival. AMD wont abandon it either, radeon is making good money today and so long as game consoles exist they will already have the architecture ready to go.

People need to stop clutching pearls over AI already.

 

[yRaz]

And some leaks suggest that most of AMD's limited TSMC silicon output will go to the server space production.
https://www.techspot.com/news/100112-leaks-suggest-there-wont-radeon-rx-8800-or.html
AI is just a buzz right now, and only thing that stops Chinese mobile-derived GPUs from conquering the World are drivers.
Allegedly (according to Moore's Law is Dead YT channel, in My opinion relatively reputable source) AMD not only didn't forced Bethesda to sabotage Starfield, but simply nVidia didn't put any resources into the game ready driver. AMD had no choice as They chosen It as their supported game (still with lots of pain), but giving as AMD is just step behind nVidia in Their ill choices, I can imagine that console support is the only thing that will make AMD drivers any half-decent in the near future, until AI craze will go away.
Start hoping Chinese will invest in game drivers developement. Americans just won't, as It doesn't bring as much money as AI.
And I'm Polish, so no big fan of dictatorships in the East.

AMD is in a position where their only advantage over nVidia in the AI space is that there are developers who can't afford A100's or H100's from nVidia so they have to. For the big companies, they just pay whatever nVidia is asking. The barrier to entry for nVidia AI products is so high that startups will have to go with "lower cost" solutions like AMD gaming cards. If AMD can make a gaming card that attracts AI developers and they're willing to deal with the shortcomings of working with AMDs platform because nVidia products are cost prohibitive, they will.

If there is anything to be said about gaming cards in the $700+ range, consumers are just not interested in paying that much to play games. If someone can build an 8 GPU AMD AI rig for $15,000 to compete with 1 of nVidias $30,000 h100 GPUs, cash strapped developers will build them. AMD cards have been sitting on shelves for literal years even with nVidia's AI machines costing what they do and a general lack of avalibity.

Also, keep in mind that AMD is making A TON of money off of nVidia right now because their GPU servers are mostly using EPYC CPUs. If anything, I think AMD would take away from GPU fab time to increase server CPU production. These ultra expensive GPUs aren't selling and most people don't need them. Maybe we'll see an 8800XT that's faster than a 7900XT for $500. That's the market everyone wants us to return to is fast and affordable GPUs so I have no problem if they don't make ultra highend to compete with things like 4090's. Go to new egg and look at the amount of Open Box 4090's for sale. It's a blazing fast card but it far from provides $1600 in value. There are only so many consumers who are willing to $7-800 for a GPU and I believe that that market has been heavily saturated. Now that interest rates are going up, people can't even easily put them on the credit card, either. I think my credit card has gone from 9% to almost 20% in the last year.

 

[noel24]

Generally, I do not disagree with You, but I'm very sceptical of Western style corporations, fighting over "quarterly numbers" and believe that They will f*ck It up just by temporary loss of interest in the consumer graphic segment, like Nvidia just did with Starfield. 30fps should not be a standard just because that *sshole Todd said so. But somehow both nVidia and AMD aim for It.

In the end I got 3 letters to tell You: I B and M.
They created the PC but are not manufacturing one today.

 

[yRaz]

Generally, I do not disagree with You, but I'm very sceptical of Western style corporations, fighting over "quarterly numbers" and believe that They will f*ck It up just by temporary loss of interest in the consumer graphic segment, like Nvidia just did with Starfield. 30fps should not be a standard just because that *sshole Todd said so. But somehow both nVidia and AMD aim for It.

In the end I got 3 letters to tell You: I B and M.
They created the PC but are not manufacturing one today.

The IBM products of the 90's were so exciting. That was an amazing time to be alive. T-1 lines and cisco 2501 routers everywhere. I don't see it as impossible, just that I see it as unlikely to happen within the next 5 years. People think of "56K" as slow, but do they remember slower than that? Actually being able to order a modem out of a magazine in the 90's to just drop in and upgrade your internet speed? I forget what game it was, but I had to upgrade from my 28K modem to a 42k(?) just to play it. I know I didn't upgrade to 56k until I bought an Athlon XP 1800+ in like Christmas of 2001, I don't remember the exact build. But jumping from a K6-2 to an Athlon XP was mind blowing. My school used all Apple products at the time and they were still running IBM Power PC CPU's.

Now, today, I look at these performance bumps and I just think, "oh, that's kinda cool" instead of having my mind blown 6-8 months.

 

[rmcrys]

Nvidia isnt going to abandon a multi billion dollar industry [...] AMD wont abandon it either, radeon is making good money today
...
People need to stop clutching pearls over AI already.

They have a max amount of available wafers, they will send the area to what makes more money. Console chips are made mostly from older nodes for many years which won't conflict too much with desktop chips. Desktop chips are in-between with highly profitable chips (3 to 5nm), so instead of doing 95% desktop GPUs (47.5% low and mid low versions, 47.5% mid and high end) and 5% AI, they may make 50/50 and from these 50%, most mid to high end to maximize the profit.

To rap up: due to emerging higher profit chips and limited availability from wafers for each, low and mid low priced GPUs are to disappear. Those not wanting to pay much will have to wait for older stocks or second hand market.

 

[Uncle Al]

I worry it also includes double the price for the users .....

 

[yRaz]

They have a max amount of available wafers, they will send the area to what makes more money. Console chips are made mostly from older nodes for many years which won't conflict too much with desktop chips. Desktop chips are in-between with highly profitable chips (3 to 5nm), so instead of doing 95% desktop GPUs (47.5% low and mid low versions, 47.5% mid and high end) and 5% AI, they may make 50/50 and from these 50%, most mid to high end to maximize the profit.

To rap up: due to emerging higher profit chips and limited availability from wafers for each, low and mid low priced GPUs are to disappear. Those not wanting to pay much will have to wait for older stocks or second hand market.

There are only so many people willing to pay $600+ for graphics cards, it's not a matter of "we're just going to make $1500 gpus now". The 7800xt sold out in the US within hours of release and the 7700XT also did very well. There are limits to what people are willing to pay. There aren't enough rich people willing to pay $1500 for GPUs to provide the second hand market with supply. I don't care how much AMD and nVidia will try to force it, there just aren't enough people to buy graphics cards at those prices.

Then game developers will turn to manufactures when their games sales drop because people won't be able to play them. "hey, nice game, I'm going paying $1500 to play it". That's just not a thing that the market can take.

 

[SalmanIsHere]

Good news: even better AI Accelerators. Bad news? Your future consumer video card will be Intel or Chinese, as both nvidia, and soon AMD slowly already forgeting how to make one.

Nvidia is not gonna leave a multi-billion dollar market. This is a fan fiction spread by amd fanboys and amd marketting channel moore law is dead.

Dekstop GPu- 84%
Laptop Gpu- 94%
Current GEn console - Switch sold more than ps5+xbox s/s combined.

Jensen recently was asked if nvidia will leave gaming- He got really pissed with that question. He said AI started with RTX 2000. We are never leaving gaming.

 

[cristianm]

"HBM4 could double memory bandwidth to 2048-bit" that is the memory bus not the bandwidth. The article is clear on that.

 

[rmcrys]

There are only so many people willing to pay $600+ for graphics cards, it's not a matter of "we're just going to make $1500 gpus now". [...]
Then game developers will turn to manufactures when their games sales drop because people won't be able to play them.

There are many ways to achieve the same goal:

- old nodes to the consoles' SoC, due to high optimization games will still run rather well
- old nodes using rebadged chips (e.g. 4070 as a 5060 with updated board) for the lower end cards, like many generations from nVidia's "MX" parts

Game developers will still sell well, only what you get from your money, performance, etc on the low end will be not so interesting.

 

[Theinsanegamer]

They have a max amount of available wafers, they will send the area to what makes more money. Console chips are made mostly from older nodes for many years which won't conflict too much with desktop chips. Desktop chips are in-between with highly profitable chips (3 to 5nm), so instead of doing 95% desktop GPUs (47.5% low and mid low versions, 47.5% mid and high end) and 5% AI, they may make 50/50 and from these 50%, most mid to high end to maximize the profit.

To rap up: due to emerging higher profit chips and limited availability from wafers for each, low and mid low priced GPUs are to disappear. Those not wanting to pay much will have to wait for older stocks or second hand market.

So new nodes will only be used for AI and newer cards, while older cards on older nodes will be rebranded to serve as mid range components.

You know, the way the GPU market worked for over a decade? All the 2000s?

Doesnt mean the GPUs will disappear. Nvidia isnt going to pull out, that would be handing a 4 billion market to AMD, who could use that money to jack up their R+D branch.

 

[rmcrys]

So new nodes will only be used for AI and newer cards, while older cards on older nodes will be rebranded to serve as mid range components.

You know, the way the GPU market worked for over a decade? All the 2000s?

Doesnt mean the GPUs will disappear. Nvidia isnt going to pull out, that would be handing a 4 billion market to AMD, who could use that money to jack up their R+D branch.

I never said it will disappear , where did you read that? And where in the 2000's you had, for example, the xx60 - xx70 (mid class) built on an inferior node than the xx80 - xx90?