Nvidia unveils GH200 super chip and DGX GH200 computing platform for AI

[nanoguy]

The big picture: Nvidia's new strategy is centered around generative AIs, large language models, and recommender systems, and so is its latest DGX supercomputer. The company believes these will soon form the "digital engines of the modern economy" as companies like Meta, Google, and Microsoft are racing to realize the benefits of AI using Nvidia's Grace, Hopper, and Ada Lovelace hardware architectures.

By now it's no secret that Nvidia has gone all-in on the idea of selling shovels to companies big and small that are manically digging in the land of generative AIs in search of digital treasure. The company is well-positioned to capitalize on this trend and could very well become the first chipmaker with a $1 trillion valuation – more than double that of TSMC, the company that makes more than half of the world's most advanced chips.

Nvidia's announcements at Computex 2023 reflect this new strategy very well. Nvidia CEO Jensen Huang has revealed the company's Grace Hopper GH200 super chips are now in full production, highlighting their potential for accelerating compute services and software for new business models and optimizing existing ones.

The new Grace Hopper super chip is a monster with 200 billion transistors

Huang says the tech industry has hit a hard wall with traditional architecture in recent years, which is why it's been increasingly turning to GPUs and accelerated computing to solve complex computing tasks. To satisfy this surge in demand, Nvidia has developed a new DGX GH200 supercomputing platform that packs 256 Grace Hopper GH200 super chips.

Each Grace Hopper unit combines a Grace CPU and an H100 Tensor Core GPU, and the DGX GH200 system is supposedly able to deliver one exaflop of compute performance as well as ten times more memory bandwidth than the previous generation. For reference, the first exascale computer was the Frontier supercomputer at the Oak Ridge National Laboratory in Tennessee, which was able to reach 1.2 exaflops in the Linmark test last year, taking the crown away from the Japanese Fugaku system.

The DGX GH200 also comes equipped with 144 terabytes of shared memory – 500 times more than in the DGX A100 system it's replacing. This should allow companies to easily build and run generative AI models like the one behind ChatGPT. Nvidia says Microsoft, Google Cloud, and Meta are among the first clients for the new supercomputer, while Japan's SoftBank is looking to bring the GH 200 super chips to data centers across the Asian country.

Nvidia will also use four DGX GH200 systems linked using Quantum-2 InfiniBand networking with up to 400 Gb per second bandwidth to create its own AI supercomputer called Helios. Separately, the company is introducing over 400 different system configurations coming to market in the coming months that integrate the Hopper, Grace, and Ada Lovelace architectures for a variety of high-performance computing applications.

When standing next to a life-size visual representation of the DGX GH200 system on stage, Huang described it as "four elephants, one GPU," since any GH200 unit has access to the entire 144-terabyte memory pool. He also tried humoring the audience by noting that he wondered if this new system could run Crysis. Given the fact that enthusiasts have been able to run the famous title right from a GeForce RTX 3090's VRAM, you could probably run many thousands of simultaneous instances using a monster like the DGX GH200.

One thing is for sure: Nvidia is almost laser-focused on capitalizing on the AI chip boom as powering advancements in this area is what brings in over half of its revenues. The new DGX supercomputer is yet another attempt to keep the industry hooked on Nvidia products. Whether a company wants to power 5G networks, generative AI services, factory robots, augmented and virtual reality experiences, or advertising engines, Nvidia wants to be the go-to vendor for all enterprises looking to use accelerated computing.

Gamers are still on the company's radar, albeit more through the lens of what AI can do to enhance gaming experiences. For instance, Nvidia's newly-announced Avatar Cloud Engine for Games will allow developers to improve interactions with non-playable characters by linking them to a large language model. The company wouldn't say what the systems requirements would look like for this new technology, but we do know Nvidia's research arm is busy exploring ways to optimize game assets in future games using AI.

Permalink to story.

https://www.techspot.com/news/98867-nvidia-unveils-gh200-super-chip-dgx-gh200-computing.html

 

[nnguy2]

Here's hoping Nvidia bets the house on AI and go bust and have to come crawling back to gamers.

 

[godrilla]

Here's hoping Nvidia bets the house on AI and go bust and have to come crawling back to gamers.

My theory is that their current strategy is by design. When the ai hype train peaks they will absolutely go back to the gamers to get to target forecast. But I wouldn't hold my breath. The only thing that can save gaming in the near term is AMD. AMD has to step up to the plate instead of hiding in Nvidia's shadow. While AMD is playing this stagnation gpu game with Nvidia I believe it will hurt their Zen 5 sales because their will be no gpu succesion that year to stretch out it legs. The next gpu succesion will be closer to Zen6. Also Nvidia already has a x86less solution for the data center While the other 2 are playing catch up.
I can't believe Huang referenced can it Run Crysis about his H100. ( Guy was cracking jokes all night) Can tell he was very excited being on stage after 4 years in CGI/remote presentations.

 

[punctualturn]

Here's hoping Nvidia bets the house on AI and go bust and have to come crawling back to gamers.

And the gamers turn their back to nvidia when that happens.

Nah, they lack a spine to do it now, imagine when nvidia releases cheaper gpus, if forced to do it.

That said, “Superchip”? Really?

Talk about being full of yourself.

 

[clockateer]

So still doesnt run crysis?

 

[m3tavision]

Sadly, NVidia shifted to scamming gamers for profit to fund other projects about 8 years ago... NV never put money back into gaming GPUs, they have been using Volta architecture and trying to sell off bits of that enterprise tech as gaming tech to unsuspecting buyers using MARKETING.

Meanwhile, AMD committed to gaming engineered a 100% gaming architecture in RDNA... for Gamers.


NV has too much going on, to worry about gaming now... and Jensen Huang admitted it himself on stage earlier this year. Suggested they spend most of their resources and time on software, not hardware...

 

[Porkous]

When we have AI even in laptops cpu (phoenix), phones and those tiny chips, nv comes with humongous chips that justify their size and power by price. Nv is too old school, or rather there is a strange trend going of legacy stocks getting inflated and staying that way?

 

[Burty117]

Meanwhile, AMD committed to gaming engineered a 100% gaming architecture in RDNA... for Gamers.

If that's true, why does Nvidia still hold the performance and efficiency crown?
AMD knew exactly what they were making their GPU's for, Nvidia apparently didn't and were laser focused on AI infused "rubbish" for the enterprise.
Why can't AMD beat Nvidia's highest end?

It's worth saying, having the fastest card matters more than anyone truly realises, every YouTuber, every review of anything will inevitably be run (and therefore advertised) on the fastest GPU. There's also the trickle down effect, "if they produce the fastest, the cards below it must be faster than the competition also", Although people are wising up to that now that pricing is through the roof.

 

[godrilla]

If that's true, why does Nvidia still hold the performance and efficiency crown?
AMD knew exactly what they were making their GPU's for, Nvidia apparently didn't and were laser focused on AI infused "rubbish" for the enterprise.
Why can't AMD beat Nvidia's highest end?

It's worth saying, having the fastest card matters more than anyone truly realises, every YouTuber, every review of anything will inevitably be run (and therefore advertised) on the fastest GPU. There's also the trickle down effect, "if they produce the fastest, the cards below it must be faster than the competition also", Although people are wising up to that now that pricing is through the roof.

AMD loves to pair their high end cpu with the 4090 for benchmarks next year for zen5 they will as the tech media to use the 4090/4090ti. It's free advertising just works. Hey did you know some people started calling Nvidia's ai artificial inflation .

 

[m3tavision]

If that's true, why does Nvidia still hold the performance and efficiency crown?
AMD knew exactly what they were making their GPU's for, Nvidia apparently didn't and were laser focused on AI infused "rubbish" for the enterprise.
Why can't AMD beat Nvidia's highest end?

It's worth saying, having the fastest card matters more than anyone truly realises, every YouTuber, every review of anything will inevitably be run (and therefore advertised) on the fastest GPU. There's also the trickle down effect, "if they produce the fastest, the cards below it must be faster than the competition also", Although people are wising up to that now that pricing is through the roof.

Do you understand why nVidia was selling pallets of the un-released RTX3080 out their backdoor to "partners" for $2,299... when the MSRP was $999..?

It's because only a few were ever sold in the gaming space early on, and those who did wait in line on opening day.. got theirs. Everything else went dry in the gaming channels, but was pooring into the professional ones. It is because nVidia didn't want it's "partners" to get all fluffed-ovr for recently buying Pro cards like the RTX5000 or 6000 and having the "$999" gaming card with 50% more compute and costing $3k less...?

It's because nVidia's value is in CUDA, not gaming.

And you are correct, having thee fastest card matters to the top 5% of Streamers, who will forgo the extra 50% in price for the 17% on-average gain ovr the much smaller Navi31.



As for Actual gaming architecture & efficiency, meant for and engineered for gaming..?

 

[JamesBlond]

My theory is that their current strategy is by design. When the ai hype train peaks they will absolutely go back to the gamers to get to target forecast. But I wouldn't hold my breath. The only thing that can save gaming in the near term is AMD. AMD has to step up to the plate instead of hiding in Nvidia's shadow. While AMD is playing this stagnation gpu game with Nvidia I believe it will hurt their Zen 5 sales because their will be no gpu succesion that year to stretch out it legs. The next gpu succesion will be closer to Zen6. Also Nvidia already has a x86less solution for the data center While the other 2 are playing catch up.
I can't believe Huang referenced can it Run Crysis about his H100. ( Guy was cracking jokes all night) Can tell he was very excited being on stage after 4 years in CGI/remote presentations.

Dont forget Intel can buy Gamer GPU designers and that will give them an edge if Nvidia leave Gamers out for a while.. Intel will see massive growth in the next 3 yrs

 

[godrilla]

Dont forget Intel can buy Gamer GPU designers and that will give them an edge if Nvidia leave Gamers out for a while.. Intel will see massive growth in the next 3 yrs

That would be music to everyone's ears and not the cringe fake ai stuff .

 

[neeyik]

Dont forget Intel can buy Gamer GPU designers and that will give them an edge if Nvidia leave Gamers out for a while.. Intel will see massive growth in the next 3 yrs

There isn't really such thing as a 'Gamer GPU' -- at an architecture overview level, there's very little difference between anything that AMD, Intel, or Nvidia produces when it comes to GPU design. All three vendors have chips with multi-format ALUs, TMUs, ROPs, matrix units, ray tracing acceleration units, and so on.

The notable differences that do exist lie in cache hierarchy and SIMD grouping, which are primarily performance aspects rather than application-specific features. That said, Intel's Alchemist layout is better suited for compute-heavy roles, which is why the current Arc cards perform better in the latest DX12/ray tracing games than in ones that predate this type of workload.

However, all these various aspects are just as important in workstation roles as they are in gaming, hence why AMD and Nvidia use the exact same chips in Radeon/GeForce gaming models, as they do in Radeon Pro/A-series professional cards. It's only when one goes into the AI/data center/compute server market that entirely different GPUs start to appear, and all three companies either have or are currently developing processors for such sectors.

 

[EdmondRC]

Nvidia having a source of reliable revenue that is greater than their gaming GPU revenue doesn't necessarily have to be a bad thing for gamers. One could argue that having larger margins elsewhere could allow Nvidia to lower its margins on gaming GPUs as long as there is capacity to produce them. Another source of revenue allows you more flexibility with pricing not less.

I'm not saying that Nvidia will see it this way, but I think it's very possible that Nvidia could indeed cut discrete consumer GPU prices more in the future, but it will still take competition to bring this about. If AMD and Intel can't bring competitive GPUs and pricing to bare, then Nvidia will have no incentive. I don't think Nvidia is going to abandon gamers even if its just 1/3 of the revenue stream in the future, but I do think they will take them for granted if they are not in risk of losing gamers to the competition.

 

[hahahanoobs]

Cool.

 

[Burty117]

Do you understand why nVidia was selling pallets of the un-released RTX3080 out their backdoor to "partners" for $2,299... when the MSRP was $999..?

It's because only a few were ever sold in the gaming space early on, and those who did wait in line on opening day.. got theirs. Everything else went dry in the gaming channels, but was pooring into the professional ones. It is because nVidia didn't want it's "partners" to get all fluffed-ovr for recently buying Pro cards like the RTX5000 or 6000 and having the "$999" gaming card with 50% more compute and costing $3k less...?

It's because nVidia's value is in CUDA, not gaming.

And you are correct, having thee fastest card matters to the top 5% of Streamers, who will forgo the extra 50% in price for the 17% on-average gain ovr the much smaller Navi31.



As for Actual gaming architecture & efficiency, meant for and engineered for gaming..?

You still didn't answer my question, why does Nvidia hold the crown? Why can't AMD beat the best from Nvidia if Nvidia aren't even concentrating on gaming?

Nvidia's scraps, the leftovers from the bottom of the enterprise barrel apparently can beat AMD's finest...

 

[m3tavision]

You still didn't answer my question, why does Nvidia hold the crown? Why can't AMD beat the best from Nvidia if Nvidia aren't even concentrating on gaming?

Nvidia's scraps, the leftovers from the bottom of the enterprise barrel apparently can beat AMD's finest...

What crown...?
Do you mean size/performance crown...? Or price/performance crown..?

Why doesn't NVidia have a chiplet GPU..? Or have the Price/performance crown..? Why is AMD's RDNA2 dies beating nVidia's brand new Ada Lovelace RTX 40 dies... for less?

Or are you asking why doesn't AMD take one if it's Pro cards... and sell it as a $1,600 gaming card..? (<---That answer is REAL simple and one that the CEO of AMD already answered.)


It's because hardly NOBODY buys $1,600 GPUs for gaming^.
Almost the exact same thing I said earlier... very few of the 4090 sales are for gaming.... they are Content Creators who do not want to buy nVidia's $4,800 PRo card.... to use with CUDA. (So @ $1,600 the 4090 is a steal...)

I already mentioned all of this. Perhaps you didn't read my last post, or understand advanced chiplet design... in which the $999 XTX beats the $1,600 4090 in several games... HOW is this possible.. if NV is wearing a gaming crown..? Even the 4080 can't beat the 4090... and the 4080 cost more than the XTX, but the XTX can beat both... 0.o

People pay top dollar, to have the top CUDA card for non-gaming.. otherwise RTX 40 is a flop.

 

[Burty117]

What crown...?
Do you mean size/performance crown...? Or price/performance crown..?

Performance, if AMD, as you say, is specifically creating their architecture for gaming, why can't they produce a product that's outright faster than the 4090?

Why doesn't NVidia have a chiplet GPU..? Or have the Price/performance crown..? Why is AMD's RDNA2 dies beating nVidia's brand new Ada Lovelace RTX 40 dies... for less?

You can try and bring the lower end chips in as much as you like, I never asked, and will never ask about the lower end chips. I'm specifically calling out AMD for not making anything as fast as the 4090. It hurts their reputation, they are defacto not the "premium" option, they won't get used in reviews of games or hardware, you won't see influencers using them, your normal everyday person who doesn't spend all day on tech websites reading everything their is to know on GPU's will simply see what the best GPU is and get a lower model that fits into their budget. Having that highest end does matter.

Or are you asking why doesn't AMD take one if it's Pro cards... and sell it as a $1,600 gaming card..? (<---That answer is REAL simple and one that the CEO of AMD already answered.)


It's because hardly NOBODY buys $1,600 GPUs for gaming^.
Almost the exact same thing I said earlier... very few of the 4090 sales are for gaming.... they are Content Creators who do not want to buy nVidia's $4,800 PRo card.... to use with CUDA. (So @ $1,600 the 4090 is a steal...)

I could be wrong, but based on just the Steam Hardware Survey, hasn't the 4090 outsold the entire 7000 series so far? Or at least, I swear that was a news article around here not that long ago. If people weren't buying them, the price would have dropped by now...

You'd get the 4090 and use it as a "professional" card if you're a one man band or trying it out at home, companies don't normally build custom computers and slap consumer grade gear in them, That's simply not how it's done.
It seems you've convinced yourself the 4090 only sold to "professionals" and are trying to pass it off as fact.

I already mentioned all of this. Perhaps you didn't read my last post, or understand advanced chiplet design... in which the $999 XTX beats the $1,600 4090 in several games... HOW is this possible.. if NV is wearing a gaming crown..? Even the 4080 can't beat the 4090... and the 4080 cost more than the XTX, but the XTX can beat both... 0.o

You mean, in very select games the 7900XTX is faster than the 4090, Why would you even mention that? The 4090 is on average, 20% faster than the 7900XTX at 4K and 17% faster at 1440p, these margins widen even more once you add ray-tracing into the mix.

AMD simply don't have the architecture to do it yet, I hope this chiplet design works out for them and in the future we finally see them return to the good old of days of beating Nvidia's top end. ATi used to be able to do it, the Radeon HD 7950 was an absolute weapon of a graphics card, I hope they can return to that state.

 

[punctualturn]

Performance, if AMD, as you say, is specifically creating their architecture for gaming, why can't they produce a product that's outright faster than the 4090?

I am genuinely confused about this one. Why do they have to release something faster than the 4090 to be considered superior?

Especially when everyone is actually buying cheap cards (according to the flawed Steam survey)?

Personally, I think AMD made a naming mistake with the 7900 models, because even though they have clearly stated that it competes with the 4080, everyone keeps comparing it to the 4090, yet conveniently ignore the price difference. Worse, as stated above, in some games, it does match the 4090 meanwhile costing up to 700 bucks less.

The truth is, nvidia simply has the mindshare of the people, to the point that they dont see any of their flaws. Yet all they see are flaws on AMD’s offerings.

Media doesn’t help, example, go to Hardware Unboxed main page and from the latest 10 videos, how many have a nvidia gpu in the thumbnail?

How many of those have “DOA” in the title?

Then do the same for AMD’s.

A buyer that doesnt know better will end up buying nvidia just by that.

 

[Burty117]

I am genuinely confused about this one. Why do they have to release something faster than the 4090 to be considered superior?

Because their isn't enough competition.
This isn't "Lamborghini vs Ferrari vs McLaren vs Bugatti vs Rimack vs Koenigsegg vs Porshe".
It's just Nvidia vs AMD and Intel maybe soon. Whoever has the fastest product is the premium choice. You could argue AMD is the premium choice vs Intel for example if you ignore Nvidia existing.
We're also ignoring how much of a gap there is between the 7900XTX and 4090, 20% is a pretty big leap when you're talking that level of performance and it's even wider when including RT.

Especially when everyone is actually buying cheap cards (according to the flawed Steam survey)?

This doesn't exist anymore, there isn't such thing as a cheap graphics card. The best value graphics card is actually the 7900XTX if you compare average framerate to price, yet its a £1000 card.

When your "low/midrange" GPU (4060/7600) is at the £400 mark, that's essentially the price of a whole PS5. You know they're massively overpriced and currently, a "cheap" GPU simply doesn't exist in the latest line-ups (yet, I for some reason still hold hope GPU pricing will eventually go down to more reasonable levels).

Personally, I think AMD made a naming mistake with the 7900 models, because even though they have clearly stated that it competes with the 4080, everyone keeps comparing it to the 4090, yet conveniently ignore the price difference. Worse, as stated above, in some games, it does match the 4090 meanwhile costing up to 700 bucks less.

Now this I can agree with, Probably doesn't help that one of the 4-5 games it's faster is Call of Duty, one of the most popular games on the planet. If all you played was Call of Duty, the 7900XTX looks faster than the 4090.

The truth is, nvidia simply has the mindshare of the people, to the point that they dont see any of their flaws. Yet all they see are flaws on AMD’s offerings.

Media doesn’t help, example, go to Hardware Unboxed main page and from the latest 10 videos, how many have a nvidia gpu in the thumbnail?

How many of those have “DOA” in the title?

Then do the same for AMD’s.

A buyer that doesnt know better will end up buying nvidia just by that.

This! This right here is why it's important AMD get a product out that's faster than the 4090, if you want to change that mindshare, if you want to get the media using your products everywhere, get loads of free advertisement, you need to create a product that is indisputably the fastest.

Nvidia's flaws are definitely seen, GeForce requiring a login, DisplayPorts being an old standard, Forcing ridiculous limits on partner cards, low amounts of VRAM, bad and outright deceitful product naming schemes, the list goes on.

But if AMD want to change the narrative, they want to get that mind share changing, they need to release a product that outright beats Nvidia's. It's worth saying, they managed it with Ryzen against Intel, No longer is AMD considered the cheaper, slower and less premium alternative to an Intel CPU, if anything, it's now the other way round.

Also worth remembering, another large part of the "mindshare" debacle are video game reviews and tests, all reviewers, streamers and testing will get done on whatever the absolute best GPU is at the time, it's been Nvidia for a long time now.
When someone asks "what's the best GPU you can get regardless of money" the answer is Nvidia.

I'm not saying everyone will rush out and buy a £1500 GPU, I'm not saying AMD will change that mindshare overnight by releasing a single product.
What I am trying to get across is, having the fastest matters, having competition at the highest end matters, and it matters more than us techies give it credit.

 

[godrilla]

I am genuinely confused about this one. Why do they have to release something faster than the 4090 to be considered superior?

Especially when everyone is actually buying cheap cards (according to the flawed Steam survey)?

Personally, I think AMD made a naming mistake with the 7900 models, because even though they have clearly stated that it competes with the 4080, everyone keeps comparing it to the 4090, yet conveniently ignore the price difference. Worse, as stated above, in some games, it does match the 4090 meanwhile costing up to 700 bucks less.

The truth is, nvidia simply has the mindshare of the people, to the point that they dont see any of their flaws. Yet all they see are flaws on AMD’s offerings.

Media doesn’t help, example, go to Hardware Unboxed main page and from the latest 10 videos, how many have a nvidia gpu in the thumbnail?

How many of those have “DOA” in the title?

Then do the same for AMD’s.

A buyer that doesnt know better will end up buying nvidia just by that.

Um the latest steam survey has the 4090 at 0.43% of total

Steam Hardware & Software Survey

Even AMD asks the tech media to use a 4090 when reviewing the 7800-x3d. Why is that? why aren't they asking them to use the 7900 xtx? Hmmm?

 

[neeyik]

Stick to the actual news topic, please.

 

[Burty117]

Stick to the actual news topic, please.

Apologies, I'd been re-reading your 7900XTX review and genuinely thought I was responding in that comment section.