What just happened? Amazon AWS made a slew of announcements this week at its re:Invent conference, many of which revolve around generative AI and how it can be used to modernize companies' services and to increase efficiency. Amazon also unveiled its next-generation chips for a wide range of cloud-based workloads and AI training models with the promise of better performance and energy efficiency.
One of the new chips is Trainium2, meant for AI model training and said to deliver up to 4x better performance and 2x energy efficiency when compared to its predecessor. It is also expected to offer 3x more memory capacity than the first-gen Trainium chips. In the press release, Amazon says that Trainium2 is purpose-built for high-performance training of foundation models (FMs) and large language models (LLMs) with up to trillions of parameters.
The company also claims that Trainium2 will be super fast, allowing programmers to train models in a fraction of the time required by the first-gen Trainium chips. According to Amazon, Trainium2 will deliver up to 65 exaflops of compute power, offering "supercomputer-class performance" and enabling customers to train a 300-billion parameter LLM in weeks rather than months.
Amazon-backed AI firm Anthropic has announced plans to use Trainium2 to train its models already.
Another new chip is the Arm-based Graviton4, which Amazon says is the "most powerful and energy-efficient AWS processor to date." It is designed for a wide range of applications running on Amazon Elastic Compute Cloud (EC2) ultraclusters, and is said to offer up to 30 percent better compute performance, 50 percent more cores, and 75 percent more memory bandwidth than Graviton3.
According to Amazon, the new chip will enable customers to improve the execution of their high-performance databases, in-memory caches, and big data analytics workloads. It can be used to process larger amounts of data faster than with third-gen Graviton chips, thereby reducing the time-to-results and lowering operating costs.
Amazon will start offering the Trainium2 chips next year, while Graviton4-powered R8g instances are now available in preview, with general availability expected in the coming months.
The new chips are going to increase competition in the AI hardware sector, which is currently dominated by Nvidia. With Microsoft also raising the heat with the recent announcement of its Azure Maia 100 chip and Azure Cobalt CPU, it will be interesting to see how the battle ramps up in the months and years ahead.