Nvidia generates up to 1,000% profit for each H100 GPU sold

[Alfonso Maruccia]

The big picture: AI acceleration is set to become one of the most prosperous hardware businesses in the upcoming months and years, and Nvidia is in a prime position to capture a significant share of this market. The H100 data center GPU is already proving to be a major revenue generator for the Santa Clara-based company.

For every H100 GPU accelerator sold, Nvidia appears to be making a remarkable profit, with reported margins reaching 1,000 percent of production costs. Tae Kim, a senior technology writer at Barron's, recently highlighted that Nvidia spends around $3,320 to manufacture a single H100 unit, which is then sold to end customers for a price ranging from $25,000 to $30,000. These estimates are provided by consulting firm Raymond James and seemingly encompass costs related to the onboard HBM memory chips as well.

If the estimations prove accurate, this could mark the onset of an unprecedented golden era for Nvidia's GPU business. The demand for H100 GPU units is so high that they are essentially sold out until 2024, Kim said. Meanwhile, AI companies are scrambling to secure sufficient GPU accelerators to fuel their generative models and AI-powered services. Foxconn predicts that the AI server market will reach a value of $150 billion by 2027, and these modern AI servers heavily rely on the robust computing capabilities of the latest Nvidia hardware.

The H100 GPU is based on the Hopper microarchitecture, designed as the data center counterpart to the Ada Lovelace architecture that empowers the latest generation of GeForce RTX gaming GPUs. However, Hopper's primary focus is not on gaming, as indicated by performance benchmarks. The H100 accelerator is equipped with a modified GH100 GPU featuring 14,592 CUDA cores, in addition to 80GB of HBM3 RAM with a 5,120-bit memory bus.

Nvidia is certainly riding a high wave in the current AI boom, although the suggested $3,320 estimated cost per H100 GPU proposed by Kim requires further clarification. Developing a new GPU is a costly and time-consuming endeavor, involving numerous hardware engineers and specialized workers. These workers require compensation, and recent estimates indicate that Nvidia's average salary for a hardware engineer is around $202,000 per year.

On the one hand, producing and selling H100 GPUs could be much more costly than the $3,320 expense reported. On the other hand, Kim has been following Nvidia for 30 years and now he's even writing a book as the "definitive history" of the company. Therefore, when it comes to Nvidia's inner working and cost levels, he likely knows what he is talking about.

Permalink to story.

https://www.techspot.com/news/99839-nvidia-garners-reported-1000-profit-each-h100-gpu.html

 

[redgarl]

Another propaganda piece to raise the stock price to absurdity... matter of fact is that the industry will push for Open Standard and Nvidia will not keep such a dominant position in Training.

As for inference, there is too many players around.

 

[Achaios]

Damn.

I day-dream daily of Jensen eating his leather jacket out of spite but instead Jensen keeps gettin' richer and richer.

Isn't there anyone who can compete with NVIDIA and reduce Jensen's bonuses?

Meet "Paul Franke"

I was in China when I read his post and I had a ton of money in my pocket to invest.

Whoever heeded "Paul Franke's" advice and bought on or abt 14 Oct 2022 is now in Early Retirement in Thailand drunk with like 10 girls around him and is laughing at us.

 

[Mister_K]

Another propaganda piece to raise the stock price to absurdity...

Surely that would be illegal?

 

[godrilla]

Another propaganda piece to raise the stock price to absurdity... matter of fact is that the industry will push for Open Standard and Nvidia will not keep such a dominant position in Training.

As for inference, there is too many players around.

Yeah along with the stock holders stating stock is going to hit $800 this year.

 

[godrilla]

Surely that would be illegal?

like insider trading for the elites oh wait!

 

[wiyosaya]

I wonder how long it will be before Jensen fully extends that middle-finger of his toward gamers?

Personally, I think the AI craze is another fad that (due to the fact that AI users have to verify that the answers it produces are not a pile of ) will wane and Jensen will go back to his attempts to gouge gamers for Nvidia GPUs.

 

[yRaz]

in Thailand drunk with like 10 girls around him and is laughing at us.

oh, those aren't girls

 

[Mister_K]

I wonder how long it will be before Jensen fully extends that middle-finger of his toward gamers?

Personally, I think the AI craze is another fad that (due to the fact that AI users have to verify that the answers it produces are not a pile of ) will wane and Jensen will go back to his attempts to gouge gamers for Nvidia GPUs.

Machine Learning what has now morphed into "AI" is certainly not a fad

 

[cristianm]

I wonder how long it will be before Jensen fully extends that middle-finger of his toward gamers?

Gamers pay for the R&D and for the volumes. Without that, those 500k GPUs for AI wouldn't bring those margins.

 

[VitalyT]

And where are the RND and marketing expenses? Kind of important to include in such article.

 

[pencea]

I wonder how long it will be before Jensen fully extends that middle-finger of his toward gamers?

I think he's already done that the moment Nvidia's flagship cards were raised to ridiculous levels.

Back then you could get the top of the GTX 8800 for $1000 or if I go further even back the Geforce 6800 Ultra was $700.

Now it costs nearly 3k to get the 4090. This is Australia dollar I may add.

 

[Dimitrios]

Anyone that is close to him please ask him to do a revision of "Leather Man" from SNL.Good example is Jimmy Fallon as the leather man in the owner of the leather store that featured Britney Spears and Will Ferrell.