OpenAI’s first in-house chip outpaces NVIDIA on speed and energy efficiency
1/29
OpenAI has published the first detailed performance figures for Jalapeño, its first chip designed specifically for AI inference. In the company’s own tests across three large language models, the chip delivered 1.5-1.9 times higher peak throughput per unit of power and 1.7-3.6 times lower end-to-end query latency than the NVIDIA GB200 and GB300 systems used for comparison. That does not mean NVIDIA has been dethroned, but it does show why the biggest AI companies increasingly want to design more of their own hardware.
Jalapeño is not competing with training chips
This is OpenAI’s first in-house inference chip, meaning it has been optimised primarily to run models that have already been trained rather than to train new large models from scratch. The company will continue to use accelerators from NVIDIA and other partners for both training and inference.
OpenAI used the public InferenceX benchmark and tested three models: GPT-OSS 120B, DeepSeek R1 670B and Kimi K2.5 1T. NVIDIA’s GB200 and GB300 were used as comparison systems.
With GPT-OSS 120B, Jalapeño achieved peak throughput of 85,448 input and output tokens per second per kilowatt, compared with 44,960 for the GB200. That gives Jalapeño an advantage of roughly 1.9 times. End-to-end query latency was 1.03 seconds versus 1.80 seconds respectively.
DeepSeek R1 produced an even larger latency gap. Jalapeño completed the query in 1.65 seconds, while the GB300 required 5.99 seconds. Peak throughput per unit of power reached 19,641 input and output tokens per second per kilowatt on Jalapeño and 11,781 on the GB300. With Kimi K2.5, OpenAI measured roughly 1.5 times higher peak throughput per unit of power and 3.4 times lower total latency for Jalapeño.
These are OpenAI’s own measurements, not results from an independent laboratory. The energy-efficiency figures also need to be read in the context of the methodology. OpenAI normalised the results using the accelerators’ published rated power rather than measuring the actual power consumption of every competing system under identical conditions.
The 700 W chip drew no more than 550 W in testing
Jalapeño has a rated power of 700 W. According to OpenAI, actual power consumption remained at 550 W or below in the workloads tested. For its calculations, the company used NVIDIA’s published rated power of 1,200 W for the GB200 and 1,400 W for the GB300.
Work delivered per unit of power is becoming increasingly important in large AI data centres. Electrical capacity, cooling and grid connections can constrain server-farm growth just as much as chip availability. If more queries can be served from each kilowatt, the infrastructure cost per generated token or user request falls.
Jalapeño’s architecture is designed around the different phases of language-model inference. Initial input processing, or prefill, is primarily compute-intensive. Generating the response token by token, or decode, depends more heavily on memory bandwidth. Time is also consumed by moving data and model state between chips and compute units.
OpenAI designed the processor, memory, networking and software as a single integrated system in order to reduce unnecessary data movement. Among other things, the architecture allows the KV cache used during response generation to remain as local as possible.
AI helped design the chip itself
AI played a direct role in Jalapeño’s development. According to OpenAI, the team moved from the initial design to tape-out in nine months. AI was used to explore implementation options, optimise arithmetic blocks and accelerate verification and measurement cycles.
The team later used Codex together with GPT-Astra to optimise the chip for three open-weight models that were not part of the original production plan. The work took two months. In selected GPT-OSS attention and mixture-of-experts blocks, AI-generated implementations ran 1.5-1.8 times faster than the earlier expert-written versions.
There is an important caveat: that performance gain applies to selected compute blocks, not to the GPT-OSS model as a whole.
NVIDIA’s problem is not Jalapeño yet, but the trend behind it
A single specialised chip will not reshape the AI hardware market overnight. NVIDIA’s advantage lies not only in the raw compute performance of the GB200 or GB300, but also in the CUDA software ecosystem, networking, server architecture and the ability to use the same platform across a wide range of models and workloads.
The significance of Jalapeño lies elsewhere. OpenAI knows exactly what workloads its own services generate and can optimise hardware around them. Google, Amazon and other major cloud companies have followed the same logic for years. OpenAI entering the market with its own chip adds further pressure to make AI compute cheaper and more energy-efficient.
OpenAI plans to deploy Jalapeño in its own compute infrastructure by the end of 2026. A second-generation chip is already deep in development, while the architecture of a third generation is taking shape.
Energy efficiency is particularly important for Europe. The growth of AI data centres means direct competition for electrical capacity and grid connections. If specialised inference chips can genuinely deliver substantially more useful work per kilowatt, their economic impact could prove more important than any single benchmark victory over NVIDIA.
