OpenAI debuts Jalapeño, its first custom AI chip to cut ChatGPT costs and reduce Nvidia dependency

Skye Jacobs

Posts: 2,010   +58
Staff
First look: OpenAI is taking the wraps off Jalapeño, a custom "intelligence processor" built with Broadcom to make its large language models cheaper and more efficient to run. The company even used its own AI models to help design the chip. Jalapeño is a purpose-built ASIC for inference, rather than for the broader mix of workloads GPUs usually handle.

By designing a chip around how tokens move through transformer architectures, OpenAI is trying to shift away from its heavy dependence on Nvidia hardware and toward a stack it controls end-to-end.

Engineering samples of Jalapeño are already running production-class workloads, including a model called GPT-5.3-Codex-Spark, while meeting the power and performance targets OpenAI set for the project. The company says early testing shows Jalapeño "will deliver performance per watt substantially better than current state-of-the-art," while Broadcom CEO Hock Tan has said it matches Nvidia's Blackwell chips and Google's Tensor Processing Units on performance.

Because the chip is aimed at inference, it can make more aggressive design trade-offs. The architecture is tuned around LLM kernels, memory movement, networking, and serving patterns rather than general-purpose compute, with the goal of improving tokens per watt on the kinds of requests that dominate ChatGPT and API traffic.

The result is less flexibility than a GPU, but potentially much better energy and cost efficiency on a narrow set of workloads that matter most to OpenAI's business.

OpenAI describes Jalapeño as the first step in a "multi-generation compute platform" that it plans to deploy in data centers by the end of 2026 and to expand over several years with partners such as Microsoft.

Broadcom will manufacture the chip and the associated server hardware, while Celestica will assemble the racks. Those systems are intended to be deployed at gigawatt scale with data center partners over multiple generations, starting in 2026.

The development timeline is part of what makes the project notable. OpenAI says Jalapeño went from initial design to tape-out in about nine months, unusually fast for a high-performance ASIC.

Internally, the company used its own models to accelerate parts of the chip design and optimization process, effectively turning generative AI onto the problem of building the silicon that will later host it.

Strategically, Jalapeño is also about reducing exposure to GPU supply constraints and price volatility, even as Nvidia continues to lead in raw performance and ecosystem depth. Tan has said the ASIC can deliver roughly 50% cost improvements versus standard AI GPUs on measures such as cost per kilowatt or cost per token, although neither company has released full public specifications or independent benchmarks.

OpenAI is arriving in a space where other large platforms are already experimenting with in-house silicon. Microsoft, Meta, and Amazon have each introduced custom chips for training or inference. Jalapeño, however, is more tightly coupled to a single provider's model roadmap and hosted services. It is aimed at OpenAI's own infrastructure rather than offered as a general-purpose accelerator like Nvidia GPUs or Google's cloud TPUs.

That choice comes with risk. A chip that is highly tuned for today's LLM architectures can be extremely efficient now, but may be less adaptable if model designs change sharply.

If Jalapeño performs as promised at scale, it could push other AI developers to think more seriously about tightly coupling model and hardware design instead of relying solely on off-the-shelf GPUs. For now, the experiment will unfold inside OpenAI's own data centers, where the company is already testing how far a focused inference ASIC can move the needle on cost, latency, and the overall experience of using large language models.

Permalink to story:

 
Mother of God please let the market move past the more generalized GPU compute architecture into specialized AI specific chips.

I'd like to buy another GPU someday without having to shell out 2/3/4x the historic cost of one for a 30% performance bump.
That's a very short-sighted remark. There'll always be a market for AI use of generalized GPUs, but if that did change, you'd see a crash on prices for current-gen hardware-- and then not see a new generation for 20+ years. There's not nearly enough revenue from videogamers to support development on cutting-edge nodes.

Long-term, it's not clear the hardware for AI and videogaming will ever diverge. Developers are already talking about a world in which game images come not from wireframe models, textures, and lighting, but directly from generative AI.
 
That's a very short-sighted remark. There'll always be a market for AI use of generalized GPUs, but if that did change, you'd see a crash on prices for current-gen hardware-- and then not see a new generation for 20+ years. There's not nearly enough revenue from videogamers to support development on cutting-edge nodes.

Long-term, it's not clear the hardware for AI and videogaming will ever diverge. Developers are already talking about a world in which game images come not from wireframe models, textures, and lighting, but directly from generative AI.

- That "doomsday" scenario didn't happen with crypto as it moved past GPUs to FPGAs and it won't happen with AI either.

It will simply allow a return to normal MSRP -> sale pricing structure over the lifetime of a product instead of the current "buy the high price now before it gets more expensive later" nonsense.

Of course AI will inform rendering, but this discussion is about the availability and pricing of hardware, not about integration of AI acceleration into consumer hardware.
 
- That "doomsday" scenario didn't happen with crypto as it moved past GPUs to FPGAs and it won't happen with AI either.
Oops! Even at its height, GPU revenues from crypto mining weren't 1% of what AI is contributing today...and even as those revenues began to drop, they were being supplanted ten-fold by AI. You couldn't possibly have chosen a worse example.

The revenue stream possible from videogamers isn't nearly enough to fund development of modern GPUs. You may not like the fact, but fact it is.
 
Oops! Even at its height, GPU revenues from crypto mining weren't 1% of what AI is contributing today...and even as those revenues began to drop, they were being supplanted ten-fold by AI. You couldn't possibly have chosen a worse example.

The revenue stream possible from videogamers isn't nearly enough to fund development of modern GPUs. You may not like the fact, but fact it is.

- It's plenty to fund modern GPUs, and was funding modern GPUs just fine right up until ~2024 or so and the release of Blackwell/RDNA4.

Not everything has to be some big gotcha fight.
 
- It's plenty to fund modern GPUs, and was funding modern GPUs just fine right up until ~2024 or so and the release of Blackwell/RDNA4.
Except this isn't correct. In 2023, only 34% of NVidia's revenue came from the gaming sector ... and even that figure is misleading, as it includes all purchase of non-professional cards, even when they're used for crypto or AI. And in fact, in 2023 gamers were already complaining card prices were high because so many "gaming" cards were being diverted to other uses. One has to go all the way back to 2019 to when gamers contributed more than 50% of GPU development cost.

Now fast-forward to 2026, where AI alone makes up 90% of NVidia's revenues, and a charitable estimate of revenue from gaming is in the 7% range ...and dropping fast.
 
Last edited:
Except this isn't correct. In 2023, only 34% of NVidia's revenue came from the gaming sector ... and even that figure is misleading, as it includes all purchase of non-professional cards, even when they're used for crypto or AI. And in fact, in 2023 gamers were already complaining card prices were high because so many "gaming" cards were being diverted to other uses. One has to go all the way back to 2019 to when gamers contributed more than 50% of GPU development cost.

Now fast-forward to 2026, where AI alone makes up 90% of NVidia's revenues, and a charitable estimate of revenue from gaming is in the 7% range ...and dropping fast.

- Ok, but that just means Nvidia gets to charge corporate AI customers more money for the same cards they were developing anyhow.

It doesn't mean Nvidia is unable to design and sell graphics modern graphics cards without AI revenue, they would just do so with a much lower margins and market cap.
 
NVidia are doomed in the long run. Making AI chips is too easy. Whilst the process nodes etc etc are highly complex that's TSMC etc. problem. The actual logic in a TPU is generally not much more than vast repeated arrays of parallel processing units for vectors, and matrices. This, combined with the simplicity of the software layer on top of this (unlike a GPU driver for instance) should make roll your own pretty straightforward. We've seen a few popping up in the last year and I can only see this continuing when companies see the eye-watering prices NVidia is asking for it's chips.
 
Just a matter of time. NVIDIA's massive profits aren't sustainable, every hyperscaler will be attempting to pivot away from them ASAP not to mention China won't want to rely on a US player in the first place.
Don't think it'll help us gamers much as ultimately it's TSMC that's the bottleneck (and Samsung/Intel to a lesser degree). But perhaps it'll get NVIDIA to at least pretend to care about gamers again.
 
Back