OpenAI 过去两年一直在低调打造“Jalapeño”，这是一款刚刚在 Hot Chips 大会上发布的推理芯片。关于其成功流片的传闻早已流传许久，但现在我们拿到了具体细节。OpenAI 邀请我们深入了解这款芯片，走进他们的实验室验证其真实程度，并用我们的 InferenceX 测试套件对它进行了基准测试。

今年 6 月，OpenAI 公布了与 Broadcom 合作的芯片项目。该芯片从零开始设计，专为 LLM 推理打造。设计工作于 2024 年年中启动，从团队初始招聘到制造流片仅用了约 16 个月，这是一个极其快速的 ASIC 开发周期。

通常来说，第一代芯片往往不具备竞争力，但 OpenAI 打破了这一惯例，达到了业界领先水平。在我们测试过的多款顶级开源模型上，它击败了所有我们能测试到的 Nvidia、AMD 和 Google 芯片。OpenAI 凭借的是极致的软硬件协同设计。令人意外的是，OpenAI 并没有在模型推理的某个特定环节过度特化，而是致力于打造一款在所有场景下都能提供高性能的通用芯片。

在这篇文章中，我们将深入介绍 Jalapeño 的架构细节、软件细节，以及它在 InferenceX 上的性能测试结果。

所有人都说 OpenAI 的芯片是专为 OpenAI 模型定制的，这种说法是错的。OpenAI 打造的是一款面向 AI 推理的通用芯片。

这个时间表令人惊叹。它证明了“利用 AI 加速芯片设计”的说法是真实的。抛开快速的开发周期不谈，OpenAI 投入了大量资金，做出了务实的设计决策，而且他们的团队实力超群，所以取得这样的成绩并不令人意外。

仅看规格参数，它就已经是一位强劲的竞争者：

而 HBM4 的采用更使其脱颖而出，足以与 NVIDIA 和 AMD 的旗舰 GPU 相媲美：

许多媒体报道这款芯片时，都引用了 OpenAI 几句随口提及的评论，声称这款芯片将以其他芯片无法企及的方式针对他们自己的模型进行优化。这种理解是错误的。Jalapeño 是一款通用推理芯片，能够运行各种模型和各类工作负载，包括我们的基准测试 InferenceX——我们是在实验室里与 OpenAI 工程师一起跑完这项测试的。OpenAI 甚至半开玩笑地向我们展示了它运行《毁灭战士》（Doom）的画面，而这款游戏仅通过 Codex 提示词就被移植到了他们的芯片上。

下面是我们最核心的能效（perf/W）结果，以每全站 utility MW（兆瓦）的 token 吞吐量来衡量。Jalapeño 完胜其他所有芯片。而且这一切都是在没有使用多 Token 预测（MTP）的情况下实现的，而图表中的其他芯片都是各自 SKU 中表现最优的配置，并且全都启用了 MTP。

Jalapeño 在几乎所有场景下的每瓦性能（perf/W）都胜过 Blackwell，而且并未针对曲线上的任何特定工作点做过调优。它不仅在低延迟场景中表现出色，在高吞吐场景中同样优秀。更公平的对标对象是单 Token 预测（Single Token Prediction）的结果，在这方面它遥遥领先于所有竞争对手。在低并发场景下，Jalapeño 展现出卓越的交互性：在 DeepSeek R1 模型上，并发为 1 时每用户吞吐量超过 700 tokens/秒。

令人惊叹的是，这一切都是通过单 Token 预测（STP）实现的——没有投机解码，也没有预填充与解码的分离部署。除了 DeepSeek R1，我们还看到了其他模型的运行情况，包括 Kimi-K2.5 和 GPT-OSS，后者达到了约 1,400 tok/秒/用户。对于所有模型，我们都确认 Jalapeño 的 GSM8k 评测结果与 Nvidia 芯片持平。

关于这些结果，有几点需要注意。首先，所有数据均由 OpenAI 提供。我们亲自在实验室验证了 InferenceX 的运行结果，但我们既没有跑完整的 InferenceX 基准测试套件，也没有看到 AgentX 的结果。在比较芯片性能时，我们更倾向于使用 AgentX，因为其数据集具有长上下文和多轮对话的特性，能够反映真实生产工作流中的缓存行为。在 8k1k 上表现良好的框架，在 AgentX 上可能表现更差，因为真实生产负载会对路由器、前缀缓存机制、缓存管理、卸载基础设施等组件造成压力。这些在单轮 8k1k 测试中都不会被考验到。更多内容请阅读我们的 AgentX 文章。

其次，我们认为与 Blackwell 的对比多少有些不完整、也有失公允。Jalapeño 真正的竞争对手是同样采用 HBM4 的 Rubin 这类芯片。Vera Rubin 系统目前正开始交付客户，而 OpenAI 手中的 Jalapeño 在一段时间内仍只会是工程样片。

因此，性能其实应该与 Rubin 而非 Blackwell 进行比较，而且在某种意义上，我们本来就预期 Jalapeño 这样的定制芯片能超越 Blackwell。正如我们在上个月分析 NVIDIA 与 CoreWeave 发布会的性能声明一文中所描述的，Vera Rubin NVL72 的每兆瓦性能（perf/MW）是 GB200 NVL72 的 5.4 倍。我们将在下文把 Jalapeño 与 Vera Rubin 七月份的性能数据进行比较。

第三，被测试的模型并不处于开放前沿。NVIDIA 和 AMD 已经使用 AgentX 发布了在 DeepSeek V4 Pro 和 Kimi K3 等更大模型上的结果。模型越大、发布越新，在新芯片上跑起来的复杂度就越高。话虽如此，OpenAI 在 Jalapeño 上运行的模型也绝非小模型。

OpenAI 以 perf/W（每瓦性能）为核心进行设计。原因很简单：OpenAI 目前的瓶颈是数据中心的电力，而不是预算或机房面积，因此每 MW 的 token 产出量至关重要。在 2026 年的 Computex 上，Jensen 表示 perf/W、可靠性和长寿命是未来 GPU 的核心特性。引用他的原话：“如果你拥有 1 吉瓦的电力，那么每瓦特的吞吐量就是收入。”他还提到，仅仅因为芯片更便宜就选择错误的架构是不明智的。

Nvidia 在 Hot Chips 2026 的 Vera 演讲中展示了同一张收入图表，并强调了这一点：“如今数据中心的限制因素是电力。”电力至关重要，并且直接驱动收入。

运营商无法简单地获取更多 MW，因为增加 GPU 与扩容电网的周期完全不同。数据中心的电力规模受限于诸多因素，例如公用电网互联、基础设施、散热能力以及 UPS/备用发电系统的设计。电网建设的延误往往远超硬件和施工的时间进度，这推动了对 BtM（表后电力，behind-the-meter）容量的需求：即建在数据中心内部的燃气轮机和现场发电机。这种容量位于电力公司的电表之后，而非从公共电网获取。它使运营商无需等待电网互联和电力公司升级就能为设施供电，这也正是 xAI 的 Colossus 2 在实际电网接入远远滞后的情况下，如此依赖 BtM 的原因。详情请参阅我们的能源模型。

正如我们在一条 X 帖子中所写，tok/s/MW 可以归约为每焦耳的 token 数，因为 1 瓦特等于每秒 1 焦耳。这使得 tok/s/MW 能够代表一个系统的效率，以及其将能量转化为 token 的能力。

在这一方面，即使与 Rubin 相比，Jalapeño 仍然胜出。OpenAI 的 Jalapeño 在每 MW 的 STP 输出 token 吞吐量上，超过了 NVIDIA 和 CoreWeave 在 7 月发布的 Vera Rubin MTP 结果，也远超 GB200 在 2025 年的 MTP 结果。正如我们在 Vera Rubin 文章中提到的，VR 之所以与 2025 年的 GB200 结果对比，是因为两者都处于类似的早期调优阶段，与 2025 年的 GB200 对比可以保持软件成熟度一致。按照这一逻辑，我们对比了 Vera Rubin 2026 年 7 月的最新结果、GB2025 年的结果，以及今天 Jalapeño 的结果。这是一个非常合理的对比：因为这些是 Rubin 最好的公开数据，而且 OpenAI 的芯片是在 Rubin 之后流片的。OpenAI 和 Rubin 目前都还不成熟，因此性能还会继续提升。

On perf/TCO, Vera Rubin and Jalapeño are head-to-head, producing almost the same number of output tokens per $. However, as previously mentioned, Jalapeño’s results are obtained without speculative decoding and Vera Rubin’s results use speculative decoding. Speculative decoding leads to a ~3-5x reduction in cost per token. When speculative decoding is implemented on Jalapeño, this will enable Jalapeño to serve tokens even more cost effectively. Of course, part of this TCO advantage comes from trading Nvidia’s high margins for Broadcom’s lower (though still high) margins. But this is not all of it. For example, Meta and Microsoft’s AI ASIC programs not getting off the ground despite being at it for much longer shows that cost is only one part of the equation. For Jalapeño’s full TCO breakdown, see the SemiAnalysis AI Cloud TCO model .

Architecturally, OpenAI chose not to disaggregate prefill and decode (PD) across separate chip pools. The draft model and main model share the same chips and fabric, a design philosophy that trades some theoretical efficiency for practical operations. The motivation is that the workload mix changes over time, for example the ratio of input to cache write to cache read to output tokens has changed significantly as we have moved through the three eras of models ( knowledge, reasoning, and agentic, as discussed in our recent article ). Therefore, picking a fixed amount of heterogenous prefill silicon and decode silicon up front can lead to inefficiencies over time. OpenAI chooses a homogenous pool in this architecture and tries to make the chip perform well on everything.

And it does. On Kimi K2.5 (which Cursor Composer 2.5 is based on), Jalapeño reaches nearly 700tok/s/user and more than 9x the next best performing chip at 100tok/s/user.

On GPT-OSS, it’s another bloodbath. Jalapeño’s iso-interactivity throughput per MW is nearly double GB200’s highest throughput point and more than 50x GB200’s concurrency 1 point. The higher concurrency Jalapeño points use EP8.

These results are impressive! However, we have to nitpick: they’re just 8k1k, a much easier workload to tune for, and there are no AgentX runs yet. As mentioned in our AgentX article, multiturn, long context workloads stress much more aspects of the serving stack, such as routers and prefix cache. Many more optimizations are needed to excel in agentic workloads. Read more about this in the AgentX article.

All these results were gathered on the A0 stepping of Jalapeño, just 9 months into the program. But there is already a B0 stepping that is currently in the fab! B0 has optimizations that deliver roughly a 25% perf-per-watt improvement over the earlier A0 silicon. Specifically, the B0 stepping delivers 13.4 PFLOPs of MXFP4 on a single reticle-sized compute die that is manufactured on TSMC’s N3P. This compares to 17.5 PFLOPs of dense Rubin NVFP4 for a single Rubin compute die that is similar size and on the same node.

This is more respectable considering Jalapeño’s TDP is only 700W compared to Rubin’s at 900-1,150W per compute die. As Jalapeño is geared towards inference rather than training, it is understandable that OpenAI doesn’t need to push TDPs higher to maximize FLOPs, but regardless the above shows that Jalapeño delivers respectable peak theoretical FLOPs.

When compared directly to other accelerators, Jalapeño has the highest HBM bandwidth per watt, and the highest FLOPs per watt, comparable to the 1,800W Rubin Max-Q configuration:

Off-package I/O is provided by an N3E I/O chiplet with 32 lanes of 800G SerDes, for the compute fabric, with 24 lanes (600GB/s) being used for local scale-up within the rack, and 8 lanes (200GB/s) for global scale-up which is the 2,048 XPU multi-rack domain. PCIe Gen 5 is used for system I/O to connect to the x86 host CPU.

Jalapeño will ship with HBM4, making this chip one of the relatively early adopters after Nvidia and AMD, even beating the established TPU and Trainium programs. As one of the key architectural principles behind Jalapeño is getting the most out of HBM bandwidth, settling for anything but the best HBM would run counter to that goal. This results in 15.4TB/s of memory bandwidth per package which bests all the other accelerators shipping that are using HBM3E. The 15.4TB/s bandwidth shows its HBM4 can hit 10Gbps pin speeds, which would give it a slight edge over the 9.6Gbps Nvidia is getting out of its HBM4 in Rubin. The HBM is likely provided by Samsung.

OpenAI taped out Jalapeño in November 2025, or more specifically, this was a tape out of the CoWoS design, not just the top die silicon. Within 9 months of that Nov 2025 tapeout, and with only 3 months of bring-up on actual silicon, OpenAI has already delivered very good results with Jalapeño. This is all the more impressive as the team is starting from zero on the software stack.

Meanwhile, Rubin’s CoWoS tape out was completed in October 2025, a month earlier, and yet the only early results we have seen are from CoreWeave’s engineering samples. Nvidia has not let us test and release benchmarks in the same way that OpenAI has, indicating their chip software is still immature. The CUDA moat is potentially dead given how fast OpenAI can bring up new models on their silicon.

They are still far from optimized and we can see that generally Jalapeño has delivered better numbers. We don’t think that Nvidia hardware is inferior, but more so that Jalapeño’s software bring-up has progressed more quickly than Nvidia’s. This speaks to the power of hardware/software co-design, which is the main area where a cracked frontier lab ASIC team can excel over more established merchant silicon players. Counterintuitively, starting from scratch may also have benefited OpenAI as it could make clean-sheet architectural decisions without worrying about backwards compatibility or older software versions.

While OpenAI has engineering samples of Jalapeño, production is currently scheduled to gradually ramp over 2027 with most of the output currently scheduled for the end of next year. For more details of unit volumes and ASPs, see the SemiAnalysis Accelerator Model .

Suffice to say, OpenAI Jalapeno is a real high volume ASIC.

When compared against Rubin’s timeline, Jalapeño’s is shockingly quick. As shown earlier, Jalapeño’s results beat Rubin’s despite Rubin’s head start.

Digging into the architecture now, the chip’s matrix engine uses MXFP numerical formats and a weight stationary systolic array, similar to TPU. But when compared directly to TPU, it has support for smaller shapes / dimensions, meaning that it doesn’t have weird performance cliffs that get exposed by awkwardly shaped matmuls on bigger systolics.

It also has 64-bit scalar cores and FP32/INT32 vector cores. OpenAI has also invested in redundancy at the tray level and has yield harvesting built in at the core and channel level. They claim that AI assistance in chip design delivered an 8% reduction in SIMD area and a 10% reduction in matrix-engine area during design. While they did not clarify the exact process/voltage/temperature (PVT) conditions, they also mentioned the AI-assisted blocks improved timing and power over the initial blocks.

The Jalapeño architecture design focuses on eliminating memory movement of KVCache and weights as well as fixed latencies and overheads in order to make it possible to get closer to the raw peak flops/bandwidth even for small batches or shapes as compared to other accelerators.

The cores and the HBM are divided into slices, where each core slice has a low-latency local view on its own slice of HBM. Synchronization between slices occurs on a high-bandwidth dedicated collective network. This minimal memory hierarchy already gives Jalapeño a big potential advantage over GPUs, where memory accesses must traverse a complicated memory system, resulting in large latencies that must be amortized or hidden over larger shapes.

This choice is feasible because with careful placement of weights and KVs, synchronization between cores can be restricted to limited, known high-bandwidth comms such as tensor-parallel communication that can be overlapped with compute.

此外还有一个通用 NoC，用于一般通信以及访问扩展（scale-up）网络。总体而言，与 Nvidia 和 Google 相比，OpenAI 通过简化的 NoC 和内存子系统节省了大量功耗，并获得了显著的性能提升。

在核心层面，OpenAI 描述的是一个乱序（OoO）核心加 L1 缓存的架构。这与其他加速器的设计模式有很大差异——其他加速器全部采用软件管理的暂存存储器（scratchpad），通常还配合异步 DMA 支持。这里的论点同样是：这让 Jalapeño 能够避免诸如栅栏（barrier）延迟之类的固定开销。在其他加速器（如 GPU）上，这些开销需要被隐藏，或通过提高单核心的工作量来摊销，从而更难逼近原始的峰值带宽/FLOPS。

其代价是 Jalapeño 必须依赖良好的预取来保证内存请求及时到达，而这在可预测性上较差，也更难推理。不过，有了处于良好工具链中的 Codex，再加上对详细追踪信息的访问，为给定形状找到具有最佳预取策略的最优内核，很可能几乎不需要人工干预。我们认为，这正是 OpenAI 能如此快速地适配 DeepSeek R1、Kimi K2.5 和 GPT-OSS 的原因。

这些核心还支持“小”矩阵维度。这意味着（取决于具体有多小）它在不同模型和批次维度上具有更好的通用性，对矩阵维度对齐、填充开销和分块（tiling）低效的敏感度更低。相比之下，TPU、Trainium 和 Etched 的芯片都采用非常大的脉动阵列，往往需要很大的批次或恰好可整除的模型维度，才能避免分块带来的效率损失。

通过 Jalapeño，OpenAI 的重点在于消除系统中的固定延迟，以便在帕累托曲线的所有区间内实现尽可能接近屋顶线（roofline）的性能。理论上，这可能使他们在多个工作点上相对 GPU 拥有优势：

但需要注意的是，即便理论上存在这样的性能上限，要在真实内核上实现这一性能可能更加困难。因此，他们的思路似乎是：

从 OpenAI 团队在 Jalapeño 上适配 InferenceX 工作负载的极快速度来看，我们对这一方法持乐观态度。

如果 Jalapeño 取得成功，这将是一个强烈的信号：业界对编程模型以及完美通用编译器的执念，正在被前沿 AI 模型所颠覆。

OpenAI writes Jalapeño kernels like assembly. Each kernel gets hand-tuned code, some running to ~3,000 lines, backed by correctness checks and a custom sanitizer. Early kernel work was human-in-the-loop rather than fully automated, but this shifted with a more scaled-up, internal version of Codex, one which OpenAI plans to pitch to enterprise customers. The internal serving engine is called “Teacup”. Interestingly, OpenAI had no internal implementation of MLA kernels until they benchmarked DeepSeek with InferenceX . The ability for Codex to write functional and efficient kernels so quickly (without any of OpenAI’s kernel engineering team intervening) shows the software pipeline’s developmental ability.

OpenAI programs Jalapeño with Gluon. Gluon is OpenAI’s kernel programming language. Built on top of Triton, Gluon preserves Triton’s SPMD (Single Program Multiple Data) programming model, but it exposes low-level programming abstractions . For example, for NVIDIA GPUs, it offers APIs that map to PTX instructions, including MMA instructions, TMA instructions, mbarrier mechanisms, and many more. The most unique abstraction Gluon provides is the layout . Generally speaking, a layout defines a mapping between a hardware resource (e.g. 5th register of warp 9) and a tensor element (e.g. tensor element on row 6 column 7). Gluon’s layout abstraction is based on Linear Layouts , a type of layout algebra OpenAI invented. Linear Layouts mathematically formalizes what a layout is and provides tools to operate on layouts. This enables many features, such as provably correct layout conversions and optimal memory swizzling.

In terms of Jalapeño’s programming model, each Gluon program maps to a persistent thread. We believe this hints that Jalapeño suits the persistent kernel programming pattern , where each program executes on multiple tiles, and the programmer, rather than the hardware scheduler, assigns the work. OpenAI mentioned TensorInfo, an abstraction that explicitly encodes layouts. This is likely the set of layouts designed for Jalapeño, which will be powered by Linear Layouts. Finally, each core offers data prefetching and decoupled out-of-order units. For example, a user might program a wait on a prefetched data, which is locked behind a semaphore.

In a weird twist of fate, OpenAI models like GPT 5.6 Sol, which currently run on NVIDIA GPUs, have been used to design a chip that poses a real threat to the CUDA moat - NVIDIA’s own GPUs are helping usher in their potential successor in real time.

Comparing across time, we can also see Jalapeño’s developmental pace, achieving more than 2x throughput improvements at certain interactivities in less than 2 weeks. Each tarball we get from the Jalapeño team has a world of wonders inside.

不仅内核性能得到了提升，在短短 8 天内，Jalapeño 团队还在之前 TP8 配置的基础上启用了 TP32，并突破了单一系统的限制，在大模型上跑通了完整的机架级配置。这样的开发速度着实令人惊叹。

为了在正式上机运行之前验证性能，OpenAI 还有一款名为 “chilisim” 的模拟器，采用固定宽度的 trace 总线，与实测硬件的误差不超过 5%。在 A0 上，trace 能力有限，但在 B0 上已大幅改善，这很可能得益于从 A0 芯片实际运行中获得的输入。工程师们演示了 Codex CLI 运行一个内部模型（绰号 “Raiku” 或 “5.3 Codex Spark”），TPOT 达到 1.2ms。

团队还展示了直接在芯片上运行的、由 Codex 编写的演示：36 FPS 的 Doom、FP32 流体动力学模拟，以及一个 “Liquid Light” 鼠标拖拽可视化。

在模型方面，OpenAI 内部的 megakernel 方案（绰号 “gigakernel”）围绕单个超大内核构建，在设备上循环执行，以降低 CPU 开销和启动时间。团队还在进一步探索测试时计算（test-time compute）策略，内部尤其关注如何有条不紊地利用 100 万次 rollout。

我们之前提到，OpenAI 在这些芯片上并没有使用 prefill 与 decode 的分离部署（disaggregation）。这让我们感到意外，因为 NVIDIA 和 AMD 的 GPU 在 PDD 下性能收益显著，即使在同构硬件上也是如此。让我们深入看看 Jalapeño 团队为什么选择这条路。

当工作负载固定不变时，prefill-decode 分离部署（PDD）看起来很有吸引力。Prefill 和 decode 对硬件的压力特性不同，因此将两个阶段分别分配到经过单独调优的资源池，可以在某个选定的输入/输出比例下提升效率。然而，生产环境的流量并不会停留在这一比例上。输入和输出序列长度、并发数、缓存命中率、投机采样接受率以及延迟目标，在一天之中都在不断变化。

一旦设备被划分为 prefill 池和 decode 池，prefill 需求过多会让 decode 芯片闲置，请求只能排队等待；而 decode 需求过多则会出现相反的情况。运维方必须持续预测正确的资源划分比例、在两侧都预留冗余容量，并不断重新平衡这个理想比例始终在变动的系统。

在统一系统中，某些资源在特定阶段可能利用不足，但每台设备始终可以用来服务下一个请求。而在分离式系统中，一块芯片可能仅仅因为被分到了错误的资源池而完全闲置。局部利用率看起来不错，但全局利用率可能很糟。

Disaggregation also breaks locality. The prefill worker produces a large KV cache that the decode worker immediately needs, so the system must transfer that state across the network before generation can continue. That adds bandwidth consumption, synchronization, queueing, and another failure domain. The cost also rises with input sequence length because KV cache grows. However, avoiding the movement of KVs is largely a power and latency optimization; being willing to move some KVs around can allow for increased hardware utilization at the expense of some power and per-request latency.

A fungible fleet shifts capacity between latency-sensitive requests and throughput-oriented batches, while a fixed split strands hardware whenever the traffic mix changes. Moreover, context length changes the balance between attention and FFN work, making any fixed hardware ratio efficient only near its design point.

The same constraint applies to speculative decoding. A draft model has to feed candidate tokens to the verifier with extremely low latency. Separating the two across specialized pools turns a tightly coupled decoding loop into a distributed protocol. The extra communication and coordination can consume the latency saved by drafting. Keeping both models on the same devices and low-latency fabric preserves the locality that makes speculation worthwhile in the first place.

However, disaggregation can still win where demand is sufficiently large, stable, and predictable, particularly when conventional GPUs need large phase-specific batches to reach good throughput. But it is not free lunch.

The Jalapeño System at the rack unit level consists of a CPU host rack and an ASIC rack. The host rack houses 16 host CPU trays named “Katsu,” each corresponding to one of the 16 ASIC trays, named “Vindaloo,” to the right of the Katsu. Each host houses two Turin-class AMD EPYC CPUs with 1.5TB of DRAM, 2x E1.S, and 2x M.2 SSDs per rack. Each tray is also specced with 400G (2x200G) frontend networking. Each Katsu tray connects to each Vindaloo tray via 8 external PCIe DAC cables that run horizontally across the rack at the front. The system level design is done in partnership with Celestica.

The ASIC rack consists of 16 Vindaloo trays and 8 scale up switch trays (6 for local + 2 for global), named “Chana.” Each Vindaloo tray consists of 8 Jalapeño ASICs, making up a total of 128 Jalapeño ASICs per rack. The ASICs are connected to each of the Chana switch trays via a copper cable backplane, just like that of Nvidia’s Oberon. The scale up topology is split into a local domain of 128 ASICs within the rack and a global domain connecting up to 16 racks or 2,048 ASICs. We will explain the bandwidth and the topology in more detail below.

Power provisioning to a sidecar host rack draws roughly 50kW provisioned (31kW in production), and the ASIC rack draws 130kW, making the total two rack system roughly 160kW. That’s basically a double-wide GB300 rack in terms of power draw.

OpenAI can connect up to 2,048 Jalapeño XPUs within a single scale-up network. The scale-up network consists of two domains, a local domain connecting all 128 XPUs over backplane within the rack, as well as a global domain connecting 2,048 XPUs over 16 racks using a hybrid of copper and optical interconnect. Each rack consists of 8 Chana switch trays. Six Chana switches in the middle are for the local domain, which come with one 102.4T Tomahawk 6 switch ASIC each. Two Chana switches at the top and bottom of the local switches are for the global domain, which we think could consist of 2x 102.4T Tomahawk 6 switches making up to 204.8T per switch tray.

In the local domain, each of the 128 Jalapeño chips has a per XPU uni-directional bandwidth of 4.8Tb/s and is connected on an all-to-all basis to 6x 102.4Tb/s Tomahawk 6 ASICs. This would amount to 48-differential pair (DP) male and female connector pairs per XPU translating to a total of 6,144DPs worth of passive copper cables per rack used for local scale-up.

For the global domain, 16 racks totaling 2,048 XPUs are connected together via a combination of copper backplane, electrical 204.8T TH6 switch, 1.6T transceivers, and optical circuit switch. Each XPU has a uni-directional bandwidth of 1.6Tb/s for the global link, which is 16-differential pair (DP) male and female connector pairs per XPU for the backplane between the XPU and the global switch. Bandwidth exiting each global switch tray of 2 ASICs each is split between the backplane and front panel optics.

Between local domain and global domain, backplane connector count per rack comes up to 64 DPs per XPU and a total of 8,192 DPs worth of passive copper cables per rack.

The global domain adopts a rail-only architecture consisting of 8-rails across the global domain. We think OpenAI routes optical links in the global domain via Optical Circuit Switches (OCS) installed in every rack. For every XPU, 1.6Tb/s of global bandwidth will travel to the global switch tray over the copper backplane. This then exits the switch through the front panel via 1.6T transceivers, which go to the passive optical switch before exiting the rack. This expands the scale-up world size to 2,048 XPUs combining 16 racks of 128 XPUs each.

Because scale-up networking is only about 10% of total system cost, that flexibility buys valuable optionality for future 10–20 trillion parameter models or 2–4 million token context windows. On deployment, OpenAI is partnering with neoclouds and is gathering reliability data with datacenter partners through January while optimizing dock-to-rack rollout time.

Next, we talk about the future of Jalapeño, whose first production token is coming soon. The next goal is 100MW, and the hurdles will mostly be hardware: How much can they produce, how well can they deploy and operate datacenters, how do they handle monitoring, and resiliency, etc. The software is already proven, and with internal models, every software headstart is easily caught up to. Behind the paywall we will discuss implications for NVIDIA, AMD, Cerebras, and other chip companies who have signed deals with OpenAI in the coming years.

We also cover production volumes, units, and timelines for the next generation chip in our Accelerator model here .