01
Low-bit numerics
FP4 is solved, ternary is not
FP4 QAT (W4A4)3.6x weight bytes; 1.5-3x end to end
Ternary / 2-bit1.0-1.3x over 2-bit kernels on CPU/GPU
KV quantizationFP8 2x; NVFP4 ~2x more
2:4 weight sparsity1.1-1.3x alone
02The survey
A roughly 1000x cut in cost and energy per solved task, at fixed quality, is credible against a 2024 baseline (a dense ~405B transformer in BF16 on H100s) but not, on current evidence, against the late-2026 frontier.
Built from research/01-report-path-to-1000x.md (edit)
Chapter 1
The "1000x" claim only means something once the baseline, the unit and the quality bar are fixed. This report measures dollars and joules per solved task at a fixed quality level on a fixed task suite, holding constant a per-user speed target (interactivity), a context-length distribution, and a power boundary that includes prefill GPUs, decode GPUs and idle power.
At near-equal scores, output per task differs 4.4x
Output tokens per task on the Artificial Analysis Intelligence Index v5 (October 2026, max effort). A cheaper token buys nothing if a model spends more of them.
The index's median output per task rose from ~35k (v4.3) to ~59k tokens (v5) in one revision.
| Model | Output tokens per task | Index v5 score |
|---|---|---|
| Claude Opus 5.5 (max) | ~119k | 58 |
| Claude Fable 5.1 | ~78k | not given |
| Claude Opus 5 | ~73k | not given |
| GPT-6 Astra (max) | ~27k | 57 |
The market trend alone gives 14x to 471x from mid-2024 to November 2026
Each published rate of price decline at constant capability, compounded over the 2.4 years since mid-2024. Against 2024, much of 1000x is what the market already does.
Derived: rate to the power of the years, as the survey computes it. These are prices, not costs, so margin compression inflates the headline.
| Estimate | Per year | By November 2026 | Years to 1000x |
|---|---|---|---|
| Epoch, constant capability | 13x | 471x | 2.7 |
| a16z, LLMflation | 10x | 251x | 3.0 |
| Open-weight Pareto (Gundlach) | 4.6x | 39x | 4.5 |
| Algorithmic only, hardware removed (Gundlach) | 3x | 14x | 6.3 |
Read the full sectionSource: research/01-report-path-to-1000x.md, lines 5 to 25
Chapter 2
Cost per solved task factors into four terms: tokens per task, bytes and FLOPs per token (weight bytes, KV bytes and arithmetic), cost or energy per byte and per FLOP (the hardware), and utilization (batching, scheduling and idle time). The physics explains why overlap is the rule.
On H100-class silicon, reading one 8-bit weight from HBM costs about 90x the two FLOPs it feeds, and the batch size at which amortized weight reads equal arithmetic energy is about 90 for 8-bit weights and about 45 for 4-bit (derived from arXiv 2607.26571). Above that batch a GPU becomes arithmetic-bound, except for KV reads, which grow with context and are never shared across sequences.
Read the full sectionSource: research/01-report-path-to-1000x.md, lines 27 to 35
Over 90% of a 2024 token's energy is moving bytes, not doing math
One generated token of the B2024 baseline (Llama-3.1-405B-class, BF16, 16 H100s, 64K context), at H100's measured 11.68 pJ per HBM bit and 0.52 pJ per tensor-core FLOP.
Derived, not measured: each token reads ~92.1 GB from HBM (its batch's share of the weights plus its own 33.8 GB of KV cache, with ~13.9 sequences fitting beside the weights). Static power is not counted.
| Term | Joules | Share |
|---|---|---|
| Weight reads (shared by the batch) | 5.45 | 58.5% |
| KV-cache reads (this sequence's own) | 3.16 | 34% |
| Arithmetic | 0.7 | 7.52% |
| Total | 9.31 | 100% |
Batching hides weight reads only until arithmetic binds
Per-token energy to read a Dense 70B model's weights from HBM, shared across a batch, against the 0.073 J of arithmetic each token needs. Above the crossover batch, fewer bits stop helping on a GPU.
| Weights | Read energy at batch 1 | Crossover batch |
|---|---|---|
| 16-bit weights | 13.1 J | 180 |
| 8-bit weights | 6.54 J | 89.8 |
| 4-bit weights | 3.27 J | 44.9 |
| 1.58-bit weights | 1.29 J | 17.7 |
Chapter 3
8 families of levers, each with what has actually been measured, what it costs in quality, and what it overlaps with. Each verdict is the survey's own heading; the full section and the note behind it hold every source.
01
FP4 is solved, ternary is not
FP4 QAT (W4A4)3.6x weight bytes; 1.5-3x end to end
Ternary / 2-bit1.0-1.3x over 2-bit kernels on CPU/GPU
KV quantizationFP8 2x; NVFP4 ~2x more
2:4 weight sparsity1.1-1.3x alone
02
MoE at ~3% activation~7x efficiency leverage; 5.2x vs naive TP serving
Activation sparsity / early exit1.5-1.8x at batch 1; ~1x at scale
03
a fast trend against a hard knowledge floor
Distillation / density~10x/yr fewer parameters at fixed benchmark
04
~50x at a million tokens, ~1x to 2x at 8K
Compressed / hybrid attention6-7x decode at 128K; ~50x KV at 1M
05
speculation buys latency, diffusion buys speed but not frontier quality
Speculation / MTP1.38x at batch 64; ~3x at 70-140 tok/s/user
Diffusion LLMs5-15x per-user speed; ~1-2x cost
06
the lever moving the wrong way
Reasoning efficiency~2x (research); 4.4x spread at frontier
Routing / effort control~8x cost for -4 pts
07
1.2x to 2.8x in six months on the same silicon
Serving software1.2-2.8x on same GB300 in 6 months
08
GPUs compound; exotic silicon mostly buys speed, not cost
GPU generation3.1x (H100 to B200); 2-5.4x (GB300 to Rubin) per MW
SRAM / wafer-scale3-6x per-user speed; cost "comparable"
Hardwired weights~50-74x per-user speed; ~1.4x cost at R1 scale
Analog in-memory~10x power (simulated)
PhotonicsNone published
Source: research/01-report-path-to-1000x.md, lines 37 to 112
Chapter 4
A useful check is the ceiling the physics allows (derived). An MoE with ~20B active parameters in FP4, batch-amortized weights and sparse KV reads does ~40 GFLOP of weight math per token, about 20x less than the baseline's 0.8 TFLOP, and at H100 coefficients its energy floor is roughly 0.03 to 0.05 J per token, about 200x below B2024's ~9 J, before any hardware improvement. Realized model-side gains land well below that ceiling because of kernel efficiency, expert-parallel communication, prefill and idle power.
Multiplied naively the levers promise a million; overlap-adjusted, about a thousand
Each lever's standalone multiplier (low to high, dot at central), grouped by the term it attacks, then the overlap-adjusted subtotals and totals. Hollow dots are the naive product of every central row.
Every compounding factor here is the survey's inference: no source has measured these levers together at frontier scale.
Table B: against B2026 (DeepSeek-V4-Pro-class on GB300 NVL72, late-2026 software), fixed late-2026-frontier quality
| Lever | Low | Central | High | Overlap treatment |
|---|---|---|---|---|
| FP4 to ~3-bit LUT or ternary experts | 1x | 1.3x | 2x | Rubin LUT is ~1.36x fewer bytes than MXFP4 |
| MoE from ~3% to ~1.5% activation | 1.1x | 1.4x | 1.8x | Sublinear efficiency leverage |
| Frontier distillation plus retrieval offload of knowledge | 1.2x | 2x | 4x | Knowledge floor; brittleness risk |
| Activation sparsity / early exit | 1x | 1.05x | 1.2x | Near zero at scale |
| KV from FP8 to FP4 / 2-bit | 1.3x | 2x | 3x | KV already ~2% of GQA baseline, so its share is small |
| Hybrid linear layers on top of compressed-sparse attention | 1.1x | 1.5x | 2.5x | Same KV term |
| Model-side per token (overlap-adjusted) | 1.2x | 2x | 5x | Naive central ~11 |
| Serving maturity on new architecture plus DSpark-class drafting | 1.2x | 1.7x | 3x | MTP already in baseline |
| GB300 to Rubin (cost) | 1.5x | 2x | 4x | Energy central ~3x |
| Hardwired / SRAM tier for routed student (cost) | 1x | 1.3x | 3x | Energy central ~2x; up to ~10x on routed share |
| Reasoning-token efficiency (cost-aware RL, adaptive thinking) | 1.3x | 2x | 4x | Frontier spread 4.4x is the existence proof |
| Routing cascade and effort control | 1.5x | 3x | 6x | Capped by hard-task share; overlaps the row above |
| Tokens-per-task subtotal (overlap-adjusted) | 2x | 4x | 10x | |
| Naive product of all central rows | 420x | |||
| Overlap-adjusted total, cost per solved task | 4x | 35x | 1,800x | High is improbable: every lever must land |
| Overlap-adjusted total, energy per solved task | 6x | 80x | 5,000x |
Table A: against B2024 (dense ~405B, BF16, H100, 2024 software), fixed 2024-frontier quality, ~64K agentic mix
| Lever | Low | Central | High | Overlap treatment |
|---|---|---|---|---|
| FP4 weights and activations (QAT) | 1.5x | 2.5x | 3.6x | Weight group |
| Sub-4-bit (LUT / ternary) beyond FP4 | 1x | 1.2x | 2x | Weight group; GPU kernels cap it |
| MoE at ~3% activation | 3x | 6x | 10x | Weight group; needs wide-EP batch |
| Capability density (data, overtraining, distillation) | 4x | 10x | 30x | Weight group; overlaps MoE on active parameters |
| Activation sparsity / adaptive depth | 1x | 1.1x | 1.5x | Weight group; erodes at batch |
| Weight-group subtotal (overlap-adjusted) | 15x | 60x | 200x | Naive central ~200 |
| Attention architecture (MLA, hybrid, compressed-sparse) | 3x | 8x | 20x | KV group |
| KV quantization (BF16 to FP8/FP4) | 2x | 3x | 4x | KV group, same term |
| KV-group subtotal (overlap-adjusted) | 4x | 12x | 40x | Naive central ~24 |
| Model-side per token (byte-share combination plus batch synergy) | 10x | 30x | 80x | Shares ~65% weights / ~35% KV at capacity-limited batch |
| Serving software and scheduling | 1.5x | 2.5x | 5x | Utilization group |
| Speculation / MTP | 1.2x | 1.6x | 3x | Utilization group; same idle compute as batching |
| Diffusion decoding | 1x | 1x | 2x | Not credited (overlaps speculation; quality gap) |
| Utilization subtotal | 1.5x | 2.5x | 4x | |
| GPU generation H100 to Rubin (cost) | 3x | 5x | 8x | Energy central ~10x; FP4 counted above |
| Specialized silicon for routed share (cost) | 1x | 1.2x | 2x | Energy central ~2x |
| Tokens per task (reasoning efficiency, routing) | 0.7x | 1.5x | 3x | B2024 is a non-reasoning model, so small reasoning replacements can spend more tokens |
| Naive product of all central rows | 1,000,000x | |||
| Overlap-adjusted total, cost per solved task | 30x | 700x | 15,000x | High requires every lever at its best |
| Overlap-adjusted total, energy per solved task | 60x | 2,000x | 40,000x | Hardware energy improves faster than $/token |
Read the full section, with tables A and BSource: research/01-report-path-to-1000x.md, lines 114 to 166
Chapter 5
Against B2026 at late-2026 frontier quality, the central estimate is ~35x in dollars and ~80x in energy, with a plausible band of ~10x to ~100x. Against whatever the frontier is when the program finishes, the advantage over a typical deployment stays roughly 10x to 50x, about the size of the cost-per-task spread already visible among top models. Reaching 1000x against the contemporaneous frontier would need a hardware discontinuity, and the only candidate, hardwired silicon at frontier scale, shows ~1.4x in cost by its own vendor's figures, not 10x.
Whether "1000x cheaper" is achievable depends mostly on which year's frontier serves as the baseline. Against 2024 it is an accounting exercise the market is already completing; against late 2026 it is out of reach for any single program on current evidence. The realistic goal is a sustained 10x to 50x lead over typical deployment at frontier quality, worth one to two years on the price curve.
Read the whole surveySource: research/01-report-path-to-1000x.md, lines 194 to 200