The Deadline Dividend

In 1946, ENIAC weighed 30 tons and occupied a gymnasium-sized room. Fifty years later, students at the University of Pennsylvania reproduced its architecture and capacity on a chip small enough to balance on a fingertip. That became computing’s familiar story: fit the same work into a smaller machine. Penn’s ENIAC retrospective

Greater density opened a different path. Engineers could keep the room and ask: how much computation can we fit inside it? In 1996, ASCI Red became the first massively parallel computer to exceed one teraFLOP. Internet companies later built a different kind of room-sized computer: a warehouse packed with coordinated servers. In 2009, Google engineers called the data center itself a massive “warehouse-scale computer.” As computation took less room, builders could spend the space they saved on still more computation. DOE on ASCI Red Google on the warehouse-scale computer

Give two endpoints ten seconds to answer the same hard question. One uses most of the interval; the other replies almost at once. The interface feels “blazingly fast,” but both endpoints still face the same deadline.

Suppose one second of the ten goes to overhead and the answer requires 500 tokens. At 159.3 tokens per second, the first endpoint can generate about 934 reasoning tokens before answering. At 1,841.5 tokens per second, the second can generate 16,074. Calculation: under constant-rate extrapolation, the synthetic reasoning budget grows 17.2 times. Artificial Analysis measured the rates. The scenario adds a one-second overhead and a fixed deadline, then assumes extra tokens do useful work. Source rates

Calling the endpoint “blazingly fast” conflates speed with latency. Speed counts output tokens per second; latency counts time from request to useful result. Fast inference offers the same choice within a deadline. A faster decoder can return fixed work sooner or spend the time it saves on more computation before the deadline. Call the useful extra work that fits before the same deadline the deadline dividend.

What the speed test proves

Artificial Analysis lists 17 gpt-oss-120b “high” endpoints under a 10,000-token prompt and rolling 72-hour medians. Fact: Cerebras produced 1,841.5 output tokens per second. SambaNova reached 705.7 and Groq 477.6; the median of the 15 endpoints reporting speed was 159.3. Calculation: Cerebras ran 2.61 times as fast as the next listed endpoint and 11.56 times as fast as the median. The comparison also recorded Cerebras’s first chunk at 0.52 seconds, first answer token at 1.60 seconds, and modeled 500-token response at 1.87 seconds. Provider comparison

Those measures describe different clocks. Time to first token includes request handling, queueing, prefill, and initial generation. Inter-token latency measures the gap between decoded tokens; its inverse gives the displayed tokens per second. End-to-end latency ends when the useful result finishes. Throughput counts work across the fleet, often with batching, while single-user speed follows one request. Price, energy, reliability, model support, and answer quality require separate measurements.

Artificial Analysis times one request at a time, starting after the first chunk. That produces a single-stream decode rate. When each token waits on the previous token, this rate determines how much generation fits before the deadline. Performance methodology

Spending the dividend

A model can spend compute after a prompt arrives to pursue a better answer. Researchers call that post-prompt work test-time compute, or TTC. Intrinsic TTC extends one serial reasoning path with more tokens, revisions, or recurrent passes; every step depends on the one before it. Extrinsic TTC surrounds the model with parallel samples, agents, tools, critics, verifiers, and a selection step. Two hundred and fifty candidates may run together, but still consume 250 trajectories and require a reliable selector.

The immediate opportunity starts with deployed models. Give a deployed model spare seconds for a longer path, more candidates, or a check. If the extra work catches an error, the system handles that task better without retraining.

Scaling changed AI because it offered a simple recipe: more parameters, data, and training compute drove loss down predictably across tested regimes. Each larger training run consumes more scarce inputs. Fact: DeepMind’s compute-optimal experiments found model size and training tokens should grow together. Epoch AI judges larger runs feasible through 2030, while naming power, chip production, data, and training latency as constraints. OpenAI’s scaling-law paper Chinchilla Epoch AI

The longer-run bet is that models can be trained to use larger inference budgets well. Pretraining commits compute before deployment; TTC spends it after a request arrives. Providers can give easy problems small budgets and hard problems large ones. Inference: if that training succeeds, TTC opens another route from compute to capability: a deployed model can improve its answer by working longer. Faster serial inference gives that work time to finish before the deadline.

Researchers have measured TTC gains in bounded settings. Interested-party claim: OpenAI says o1 and o3 improved with longer thought. Snell and colleagues made prompt-adaptive TTC over four times as efficient as best-of-N on tested math problems; in some FLOPs-matched cases, a smaller PaLM 2 model beat one roughly 14 times larger. Best-of-250 sampling raised DeepSeek-Coder-V2-Instruct’s SWE-bench Lite coverage from 15.9% to 56%. Returns varied by task, budget, and verifier. OpenAI on o1 OpenAI on o3 Snell et al. Large Language Monkeys

Cerebras already packages this idea as CePO. CePO plans a solution, generates candidates, compares inconsistencies, and chooses through structured Best-of-N. Interested-party claim: Cerebras reports 10 to 20 times one-shot token use, about 100 tokens per second on Llama 3.3-70B, and scores above Llama 3.1-405B on selected benchmarks. No independent matched test confirms those comparisons. OptiLLM can send the workflow to several providers, so customers can switch without abandoning it. Inference: because customers can route CePO elsewhere, Cerebras must use its speed to offer a better result for the money. CePO results CePO documentation OptiLLM

When inference becomes execution

Karpathy’s “Software 3.0” describes natural-language prompts as programs executed by an LLM. “Software Is Changing (Again)” Software becomes execution when a model repeatedly decides, acts, observes, and verifies. Each call receives the state produced by the previous call, placing model latency on the program’s critical path.

Consider eight steps:

plan → search → inspect → edit → test → diagnose → retest → report

Give each step 0.75 seconds of fixed overhead, 1,000 reasoning and action tokens, and a 1.5-second tool round trip. Calculation: at 159.3 tokens per second, the loop takes 68.2 seconds. At 1,841.5, it takes 22.3 seconds, a 3.05-times end-to-end gain from an 11.56-times decode-rate gain. Tools and fixed overhead absorb part of the raw advantage. Reliability compounds too: with an assumed independent 95% success rate per step, all eight steps succeed 66.3% of the time.

Eight actions require eight model turns. Inference: faster turns could let a product complete more cycles before the user or environment moves on, increasing inference demand.

OpenAI reports work on model reasoning and serving delay. Interested-party claim: it calls GPT-5.6 Sol its flagship; max allows more thought, while ultra adds subagents. OpenAI publishes a 750-token-per-second ceiling for Sol on Cerebras, compared with more than 1,000 for the smaller Codex-Spark model. Those are OpenAI’s figures, not an independent matched test. The company also reports cutting client-server overhead by 80% and time to first token by 50%. OpenAI on GPT-5.6 Sol OpenAI on Codex-Spark

Exhibit: when raw speed survives the clock

Fixed overhead, tools, verification, and completion caps leave fewer seconds for model work. The calculation below shows how many reasoning tokens still fit.

For one serial path, let

T is latency tolerance; F is fixed request, queue, prefill, and first-token overhead; B is serial reasoning tokens; A is answer tokens; s is decode rate; U is tool time; and V is verification or selection time. With completion cap C, the available serial budget is B = max(0, min{s(T-F-U-V)-A, C-A}). Holding overhead fixed, fitting m times the baseline reasoning budget requires

Parallel reasoning has different accounting. With enough capacity to run every branch at once, wall-clock generation follows the slowest required branch, while total compute sums tokens across all branches. The table isolates intrinsic serial work by setting U and V to zero; the execution loop above adds tool time explicitly. Artificial Analysis supplies the rates. The latency tolerance, overhead, 500- or 1,000-token answer, four-times target, and constant-rate extrapolation are assumptions.

The two-second case gains 2.78 times the reasoning room and misses the fourfold target. The ten-second case gains 17.21 times. At 60 seconds, Cerebras’s published 40,960-token completion cap cuts the extrapolated reasoning budget from about 107,650 tokens to 39,960 after the assumed answer. Cerebras model metadata

From a fast endpoint to a market position

Interested-party claim: Cerebras attributes its speed to WSE-3, which puts 900,000 cores, 44 GB of SRAM, and 21 PB/s of memory bandwidth on one wafer. Cerebras says the design keeps compute, memory, and communication close rather than repeatedly moving model state among separate devices. Cerebras prospectus Artificial Analysis cannot tell how much of the measured rate came from hardware, quantization, software, scheduling, or endpoint configuration. The endpoint result nonetheless fits the architectural claim.

Fact: OpenAI has contractually committed to buy 750 MW of Cerebras inference capacity and related services over a multiyear term. It also supplied an approximately $1.0 billion working-capital loan for the buildout and holds an option for another 1.25 GW by the end of 2030, for up to 2.0 GW in total. OpenAI says the first 750 MW will arrive in phases through 2028 as a dedicated low-latency component of a workload-matched compute portfolio. Cerebras Q1 10-Q OpenAI partnership

Amazon says Bedrock will run parallel, compute-heavy prefill on Trainium and serial, memory-bandwidth-heavy decode on Cerebras CS-3. The June 2026 commercial agreement includes an initial multiyear lease of Cerebras systems, pricing commitments, exclusivity, options for future purchases, and minimum manufacturing-capacity guarantees. AWS announcement Cerebras Q1 10-Q

OpenAI uses Cerebras for low-latency Codex-Spark. AWS plans to send Bedrock’s serial decode to Cerebras and prefill to Trainium. Inference: Cerebras can earn from latency-sensitive decode even when customers use GPUs for training, batched serving, or models Cerebras does not support.

Cerebras lists output at $0.75 per million tokens, against a provider median of $0.60. Calculation: the premium is 25%. At equal token spend, four times the base reasoning would require the fast endpoint to charge no more than 0.339 times the baseline price, so brute-force sampling does not become cheap merely because it becomes fast. Token price alone misses the economics. The premium can pay if extra computation lowers the cost or raises the value of a correct, timely result. Count prompt processing, caching, tool calls, verification, failed attempts, and reserved capacity. None of the cited studies directly measured endpoint energy; TokenArena models it. Provider pricing TokenArena

Inference: Cerebras has its largest advantage where serial decode dominates the critical path. SambaNova and Groq hold the next two speed positions. GPU providers can favor batching; application builders can shorten completion time with smaller models, caching, routing, parallel calls, and faster tools. OpenAI’s own pipeline work cut round-trip and first-token delays. For jobs that still wait on serial generation, Cerebras’s measured rate cuts the largest term. Provider comparison OpenAI on Codex-Spark

The commitments are large relative to the current business. Fact: in the quarter ended March 31, 2026, Cerebras recorded $193.4 million of revenue, including $82.8 million from cloud and other services; companywide GAAP gross margin was 45%. It also reported $25.0 billion of remaining performance obligations, a significant amount from OpenAI. SEC-filed Q1 results Cerebras Q1 10-Q

Cerebras expects to recognize about 16% of that backlog through March 2028. Calculation: about 84% falls after that date under the company’s current schedule, alongside the phased capacity buildout. OpenAI anchors a significant amount, and the initial 250 MW includes data-center pass-through costs. MBZUAI and G42 supplied 63% and 11% of Q1 revenue, while TSMC fabricates all Cerebras wafers. Cerebras Q1 10-Q

Product breadth presents a different test. Artificial Analysis tracks six Cerebras configurations across three open-weight model families and scores this endpoint at 87% accuracy versus SambaNova’s 98% in the snapshot. Inference: adding models and closing that fidelity gap would let Cerebras compete for more work, provided its speed survives loaded p95. Positive service contribution margin would show that paid use covers the direct costs counted. Provider comparison Cerebras catalog

Inference: speed can change how much inference customers buy and what each result is worth. CePO illustrates heavier token use; the eight-call loop illustrates more frequent model use. Training may eventually teach models to benefit from still more post-prompt compute, though the cited studies do not establish that scaling curve. Cerebras can earn inference revenue even when other companies supply the model or application.

By the end of 2028, investors should be able to test this case on at least one important serial workload. Cerebras should beat the best matched-quality commercial alternative by at least twofold at p95 end-to-end latency under the same output contract and disclosed concurrency. An ablation should identify decode as the largest reducible delay, and the added post-prompt compute should lower all-in cost per correct result relative to that same alternative. Paid use should also produce a positive service contribution margin. Miss that combination, and public evidence will not support a large-share claim.

ENIAC’s descendants followed two paths. One carried yesterday’s computation into a pocket. The other used greater density to put more computation inside a building. Fast inference offers the same choice within a deadline: return fixed work early or use the saved time for more reasoning and action.

No rival in the cited gpt-oss-120b snapshot matches Cerebras’s rate. Provider comparison CePO and Codex-Spark show two ways to sell speed: comparing candidates and shortening interactive coding turns. Neither has a matched public benchmark demonstrating the same lead. CePO results OpenAI on Codex-Spark OpenAI quotes a 750-token-per-second ceiling for Sol on Cerebras and says Codex-Spark delivers more than 1,000 tokens per second, and it has committed to a larger Cerebras buildout. Amazon plans Bedrock distribution. These are company announcements, not matched independent tests. OpenAI on GPT-5.6 Sol OpenAI partnership AWS announcement

Two inferences follow from today’s evidence. Cerebras’s speed can give current systems more room to reason before a deadline and serial software more turns to act. A third opportunity is speculative: future models may learn to exploit larger TTC budgets. Under the exhibit’s fixed-overhead, constant-rate assumptions, Cerebras’s measured gpt-oss-120b rate leaves more reasoning-token room before the deadline than any other endpoint in that snapshot. Customers may pay for the shorter wait. They may pay more when the saved time lets the system solve a harder problem or take another action.

Linked source notes

Evidence key. Fact marks directly reported data within its stated scope. Interested-party claim flags a vendor, customer, or issuer with commercial incentives. Calculation follows displayed assumptions. Inference interprets evidence. Speculation names a testable possibility. Sources were opened and accessed August 11, 2026; live benchmark values are snapshots.

1. Historical scope: Penn supplies ENIAC’s dimensions and the teaching-chip comparison; DOE supplies ASCI Red’s 1996 terascale milestone; Google defines the warehouse-scale computer. Penn DOE Google Research
2. Benchmark scope: Artificial Analysis measures single-prompt endpoint performance and rolling medians. Same model names can hide configuration and accuracy differences. Provider comparison Methodology
3. Research scope: Chinchilla and Epoch describe pretraining inputs and feasibility. TTC evidence covers only cited models, tasks, verifiers, and budgets; it supports a scaling candidate, not an unlimited law. Chinchilla Epoch Snell et al. Brown et al.
4. Commercial scope: OpenAI, Amazon, and Cerebras are interested parties; SEC filings supply contract and financial facts where available. OpenAI partnership GPT-5.6 general launch GPT-5.6 preview Amazon 10-Q
5. Reproducibility: The exhibit uses full-precision endpoint values; displayed inputs and outputs are rounded. Cerebras model metadata supplies the completion cap. The exhibit and loop calculations can be reproduced from the assumptions printed beside them. Provider table Model metadata
6. Currentness: As of the access date, Cerebras had reported Q1 and scheduled Q2 results for August 12. Cerebras investor calendar