Amazon's Graviton3 CPU has 7 chiplets, 64 cores, and tri-socket motherboards

[mongeese]

In a nutshell: After being announced six months ago, Amazon has finally dropped their third-gen Graviton CPUs into their rentable AWS instances. Amazon's also revealed their one very unique feature: exactly three of them slot into one motherboard, no more and no less.

Amazon's Graviton series is the line of Arm processors that it designs and produces itself, for itself. It makes them available virtually through AWS (Amazon Web Services) to the public and companies of any scale; Amazon has proudly announced that Epic Games, Formula 1, and Twitter have been testing and liking Graviton3 instances ahead of last week's release.

Amazon hasn't gone into much detail with the specs, only saying that the Graviton3 has 64 cores in its compute chiplet. With all its chiplets combined, it has 55 billion transistors, almost twice as many as the Graviton2 and more than even the 64 core AMD Epyc Rome CPU.

You can spot the seven chiplets in the diagram below. In the middle is the monolithic compute die with all the cores. South of it is a pair of PCIe 5.0 controllers and flanking the sides are four DDR5 controllers that, based on the diagram, manage two lanes each.

Amazon says that the Graviton3 is 25% faster per-core than its predecessor from 2020. In all-core scenarios, its floating-point and cryptography throughput are doubled, and its machine learning performance is tripled. Not bad, although the Graviton2 had just a quarter of the cores: 16.

AWS C7g instances using the Graviton3 are now available with up to 64 vCPUs, or one whole processor. In its maximum configuration, the system has 128 GB of DDR5 and 30 Gbps of network bandwidth. Instances are available all the way down to 1 vCPU (core), as usual.

ServeTheHome notes that Amazon is treating each processor as a separate node, which might be why they're able to put three on a motherboard. And the how: instead of using a parallelized approach that would lend itself to two or four sockets, each motherboard is wired into a Nitro card that manages all the networking, security, and storage.

It's a shame that Amazon won't sell the Graviton3 and its chipset because it would be awesome to fool around with. AWS can't stop going from success to success.

Permalink to story.

https://www.techspot.com/news/94750-amazon-graviton3-cpu-has-7-chiplets-64-cores.html

 

[Dr Roboto]

Can anyone share the business case for making their own CPU? It does not seem like the cost is worth it.

 

[kiwigraeme]

Can anyone share the business case for making their own CPU? It does not seem like the cost is worth it.

I know stuff all about the server space .
The question to answer yours does or will Google ,Microsoft or Apple design their own server chips.

Google, Amazon and Microsoft are huge in the server space and are exceedingly rich.

What I will guess is that if Microsoft want something they will collab with Intel , AMD , or some ARM company aka Xbox - fool the company who don't panda ( better spelling ) to the big guys

 

[Dr Roboto]

I know stuff all about the server space .
The question to answer yours does or will Google ,Microsoft or Apple design their own server chips.

Google, Amazon and Microsoft are huge in the server space and are exceedingly rich.

What I will guess is that if Microsoft want something they will collab with Intel , AMD , or some ARM company aka Xbox - fool the company who don't panda ( better spelling ) to the big guys

But I can't understand how buying AMD or Intel server chips would not be more cost efficient. What does Amazon's chip do so much better than AMD/Intel that it requires building their own? I do understand that they are using ARM and that is a design you can just buy, but still.

 

[Mr Majestyk]

The gains over the Graviton 2 seem mild in some cases. I doubt AMD would worry too much with Epyc getting AI accelerators and the such going forward. Turin and Venice are going to lay some serious smack down on the rest of them come Zen 5 and Zen 6, with 256 and 512 cores.

 

[Vanderlinde]

Can anyone share the business case for making their own CPU? It does not seem like the cost is worth it.

Ofcourse it does (and it's worth it).

General purpose CPU's like X86 or X64 in today's case are good at everything, but do not excell in particular workloads. A GPU is extremely good at certain operations, but not so good at general operations. They design their own CPU for pretty much running their own optimized code or platform to get the best out of their situation.

And indeed these setups can cut costs significant. We're talking in the hundreds of thousans and you have your own IP and no longer "dependend" of Intel, AMD or another party really.

 

[punctualturn]

Can anyone share the business case for making their own CPU? It does not seem like the cost is worth it.

Lots of info in this link, which might answer your question:

https://www.phoronix.com/scan.php?page=article&item=graviton3-amd-intel&num=1

 

[human7]

Can anyone share the business case for making their own CPU? It does not seem like the cost is worth it.

TL;DR: Their marketing department would say this: You can get Intel/AMD anywhere. But AWS has Graviton. Google Cloud, Azure, etc, don't. That's why you should choose AWS.

Long version:
It's all about competitive advantage and cost efficiency. Current benchmarks of the c7g.4xlarge (graviton) vs the c6i.4xlarge (intel x64) put them at comparable performance benchmarks, with Graviton3 pulling ahead slightly. But you can get Intel or AMD hardware anywhere, you can only get Graviton with AWS.

AWS is playing two long games here: 1, that ARM hardware is going to make more inroads into the server space, and 2, that they can offer hardware that is unique to them. Even if the latter doesn't prove that attractive, the former seems to be working, and the power efficiencies of ARM mean they can offer similar price compute at either higher margins or lower cost, depending on how they decide to price it.

There are workloads that are best suited to x64, but many are equally suited to ARM.

Others have said that AMD has great scalability, which is true and remains so, but the maximum size AWS is offering now is a 32xlarge instance for Intel and 48xlarge for AMD (128 and 192 CPU threads, which is 64 and 96 physical cores, expected to be operated across multiple sockets). The point is this: the scalability of AMD isn't as important when cloud providers aren't offering all of it (they have to commit to specific hardware configurations across the globe for quite a long time, after all).

In other words, if AWS has to buy a billion CPU cores anyways, designing their own hardware isn't much of an expense, and it can pay dividends. The next generation of Intel and AMD hardware may come out this year, but it's unlikely we'll see it show up at AWS for another year or two, at best, unless the gains are absolutely stellar, due to the long term commitments and planning that goes into offering standardized instance types across the globe.

https://www.phoronix.com/scan.php?page=article&item=graviton3-amd-intel&num=9

 

[OortCloud]

Can anyone share the business case for making their own CPU? It does not seem like the cost is worth it.

They all launched ****-rockets last year now they are all making chips with their own names on. Next year they will all build underwater homes and marry themselves.

 

[sorten]

"It's a shame that Amazon won't sell the Graviton3 and its chipset because it would be awesome to fool around with"

If you're just looking to "fool around" with a bit of tech for a while, then renting from AWS is the perfect choice for you. Someone else has already set up the hardware for you, and you only have to pay for actual use. Then when you get bored and realize it's just a tool for getting work done, you can turn off the instance and stop paying for it. Much more appealing that spending hundreds or thousands of dollars for the same experience.

 

[Steveb8189]

Can anyone share the business case for making their own CPU? It does not seem like the cost is worth it.

Vertical integration. Just take as many players our of your supply chain / production process as possible to increase efficiency and reduce cost.

 

[zamroni111]

Can anyone share the business case for making their own CPU? It does not seem like the cost is worth it.

As graviton is pure compute with minimalist io, Amazon can easily use arm base design

 

[Dr Roboto]

Thanks to everyone that replied to my questions. Super helpful!

 

[maxxcool7421]

Can anyone share the business case for making their own CPU? It does not seem like the cost is worth it.

Intel and AMD big iron come with huge long term EXPENSIVER contracts. ARM from a single vendor is the same ...

Now they can build the core to accelerate specific chores versus general purpose functions which lend well to cloud acceleration. Case in point, the grav-3 cores themselves have almost NO I/o ability freeing up cycles to do their primary jobs. I/o is handled elsewhere..