Research library
The research, in full
Stacked levers reach 1000x only against 2024
A roughly 1000x cut in cost and energy per solved task, at fixed quality, is credible against a 2024 baseline (a dense ~405B transformer in BF16 on H100s) but not, on current evidence, against the late-2026...
6,727 words · 01-report-path-to-1000x.mdThe designCorerope design: inventing a 1000x cheaper frontier LLM
Formerly titled "Thousandfold". Written Nov 19, 2026. Built on the companion report (01-report-path-to-1000x.md) and the experiments in /experiments.
3,935 words · 02-corerope-design.mdResearch noteLow-bit numerics for LLM inference efficiency (quantization, QAT vs PTQ, pruning/N:M sparsity), state as of Nov 2026
Scope note: research done 2026-11-18. Dates are marked per finding. "Older" means pre-2025. Vendor claims are flagged as such. Several primary sources (BitNet b1.58 2B4T arXiv paper and HF model card) could not...
4,094 words · quantization_numerics.mdResearch noteSparse and Conditional Computation for LLM Inference Efficiency (MoE, activation sparsity, early exit, layer skipping, mixture-of-depths)
Research date: 2026-11-18. Older (pre-2025) results are marked [OLDER]. Some 2026 model specs come from secondary aggregators and are marked [SECONDARY].
2,916 words · moe_conditional_compute.mdResearch noteAttention Alternatives and KV-Cache Efficiency for LLM Inference (state as of Nov 2026)
Notation: "[older]" marks pre-2025 results. "[not re-fetched]" marks well-known paper claims cited from the paper's abstract/arXiv record that were not re-opened during this research session (verify before...
3,038 words · attention_kv_cache.mdResearch noteDecoding Efficiency and Test-Time Compute Economics (state as of Nov 2026)
Scope: speculative decoding (EAGLE-3, Medusa, draft models, lookahead), multi-token prediction (DeepSeek MTP and successors such as DSpark), diffusion LLMs (Mercury 1/2/2.5/3, Gemini Diffusion, DiffusionGemma,...
3,674 words · decoding_test_time.mdResearch noteCapability Density, Distillation, and Inference Cost Trends for LLMs (as of Nov 2026)
Every major estimate puts the price decline at constant capability at roughly 5x to 40x per year, with the most recent and most careful estimate (Epoch, Sept 2026) at about 13x/year (47% per quarter) since...
3,317 words · distillation_cost_trends.mdResearch noteInference Hardware and Serving Systems: Energy and Cost per Token (state as of Nov 2026)
Notes on conventions: "Vendor claim" = stated by the company selling the hardware. "Independent" = SemiAnalysis InferenceX/InferenceMAX, EE Times hands-on demo, or academic measurement. "Audited" = MLPerf...
3,861 words · hardware_systems.mdRecorded resultsRecorded experiment results (Nov 19, 2026)
338 words · RESULTS.mdEvery source cited
130 distinct links from 62 sites, with each place they are cited. Labels in the notes say whether a figure was measured independently, claimed by a vendor, or derived.
- a16za16z.comCited in the survey line 23, distillation and cost trends line 33, distillation and cost trends line 34, distillation and cost trends line 35, distillation and cost trends line 129, distillation and cost trends line 146
- ACL Anthologyaclanthology.orgCited in distillation and cost trends line 107
- AI Weeklyaiweekly.coCited in the survey line 43, low-bit numerics line 25
- Alibaba Cloud / Qwen blogalibabacloud.comCited in attention and kv cache line 13, attention and kv cache line 92, attention and kv cache line 200
- AI2allenai.orgCited in the survey line 65, attention and kv cache line 99
- alphaXiv 2510.26692alphaxiv.orgCited in the survey line 65, attention and kv cache line 14, attention and kv cache line 94
- alphaXiv 2603.23701alphaxiv.orgCited in the survey line 55, moe and conditional compute line 127
- alphaXiv 2605.25745alphaxiv.orgCited in the survey line 77, decoding and test-time compute line 136
- alphaXiv 2605.26297alphaxiv.orgCited in the survey line 31, attention and kv cache line 135
- aman.aiaman.aiCited in the survey line 51, moe and conditional compute line 14, moe and conditional compute line 76, moe and conditional compute line 156
- AMD blogamd.comCited in low-bit numerics line 150
- Analytics India Magazineanalyticsindiamag.comCited in the survey line 69, decoding and test-time compute line 24
- Artificial Analysisartificialanalysis.aiCited in the survey line 75, decoding and test-time compute line 121, decoding and test-time compute line 166
- Artificial Analysisartificialanalysis.aiCited in the survey line 7, decoding and test-time compute line 122
- Artificial Analysisartificialanalysis.aiCited in the survey line 7, the survey line 59, distillation and cost trends line 78, distillation and cost trends line 79, distillation and cost trends line 80, distillation and cost trends line 81
- Artificial Analysisartificialanalysis.aiCited in the survey line 11, the survey line 51, moe and conditional compute line 13, moe and conditional compute line 79, moe and conditional compute line 141
- Artificial Analysisartificialanalysis.aiCited in the survey line 9, distillation and cost trends line 148
- arXiv 2312.04927arxiv.orgCited in attention and kv cache line 109
- arXiv 2402.01032arxiv.orgCited in attention and kv cache line 109
- Jamba paper (arXiv 2403.19887)arxiv.orgCited in attention and kv cache line 29
- DeepSeek-V2 paper (arXiv 2405.04434)arxiv.orgCited in attention and kv cache line 61, attention and kv cache line 168
- MagicDec arXiv 2408.11049arxiv.orgCited in decoding and test-time compute line 20
- arXiv 2408.14690arxiv.orgCited in the survey line 55, moe and conditional compute line 100
- DeepSeek-V3 tech reportarxiv.orgCited in decoding and test-time compute line 23
- arXiv 2502.05172arxiv.orgCited in moe and conditional compute line 35
- arXiv 2502.08606arxiv.orgCited in the survey line 59, distillation and cost trends line 106
- arXiv 2503.01840arxiv.orgCited in the survey line 69, decoding and test-time compute line 16
- arXiv 2504.01296arxiv.orgCited in the survey line 77, decoding and test-time compute line 132
- arXiv 2504.03624arxiv.orgCited in the survey line 65, attention and kv cache line 27, attention and kv cache line 100, attention and kv cache line 199
- Cartridgesarxiv.orgCited in the design line 128, the design line 195
- arXiv 2507.06457arxiv.orgCited in attention and kv cache line 91
- arXiv 2604.10852arxiv.orgCited in the survey line 87, hardware and serving line 38, hardware and serving line 97
- arXiv 2406.05955arxiv.orgCited in the survey line 55, moe and conditional compute line 101, moe and conditional compute line 108, moe and conditional compute line 166
- arXiv 2408.03314arxiv.orgCited in the survey line 77, decoding and test-time compute line 161
- arXiv 2410.06270arxiv.orgCited in moe and conditional compute line 158
- arXiv 2411.04330arxiv.orgCited in the survey line 45, low-bit numerics line 29, low-bit numerics line 30, low-bit numerics line 31, low-bit numerics line 92, low-bit numerics line 109, low-bit numerics line 110
- arXiv 2412.04315arxiv.orgCited in the survey line 23, the survey line 59, distillation and cost trends line 56, distillation and cost trends line 57, distillation and cost trends line 58, distillation and cost trends line 59, distillation and cost trends line 60, distillation and cost trends line 62, distillation and cost trends line 63, distillation and cost trends line 126
- TerEfficarxiv.orgCited in low-bit numerics line 68, low-bit numerics line 152
- arXiv 2507.17702arxiv.orgCited in the survey line 51, the survey line 170, moe and conditional compute line 34
- arXiv 2508.10925arxiv.orgCited in the survey line 51, moe and conditional compute line 22, moe and conditional compute line 78, moe and conditional compute line 154
- Diaz, 2025arxiv.orgCited in the design line 128
- arXiv 2509.25149arxiv.orgCited in the survey line 41, the survey line 170, low-bit numerics line 12, low-bit numerics line 13, low-bit numerics line 14, low-bit numerics line 15, low-bit numerics line 16, low-bit numerics line 63, low-bit numerics line 111, low-bit numerics line 149
- arXiv 2511.19495arxiv.orgCited in the survey line 59, distillation and cost trends line 124, distillation and cost trends line 125
- arXiv 2511.23455arxiv.orgCited in the survey line 23, distillation and cost trends line 18, distillation and cost trends line 19, distillation and cost trends line 20, distillation and cost trends line 21, distillation and cost trends line 22, distillation and cost trends line 23, distillation and cost trends line 24, distillation and cost trends line 25, distillation and cost trends line 26
- Nemotron 3 paper (arXiv 2512.20856)arxiv.orgCited in attention and kv cache line 28
- Engram paperarxiv.orgCited in the design line 42, the design line 193
- Mamba-3 paper (arXiv 2603.15569)arxiv.orgCited in attention and kv cache line 37
- arXiv 2310.02410arxiv.orgCited in moe and conditional compute line 164
- arXiv 2501.12948arxiv.orgCited in the survey line 59, distillation and cost trends line 83
- Stop Overthinking survey, arXiv 2503.16419arxiv.orgCited in decoding and test-time compute line 137, decoding and test-time compute line 138
- arXiv 2505.13417arxiv.orgCited in the survey line 77, decoding and test-time compute line 131
- Ling 2.0 paper, arXiv 2510.22115arxiv.orgCited in moe and conditional compute line 24, moe and conditional compute line 157
- search result: Kittyarxiv.orgCited in low-bit numerics line 93
- arXiv 2602.09113arxiv.orgCited in the survey line 69, decoding and test-time compute line 34
- arXiv PDFarxiv.orgCited in moe and conditional compute line 109
- arXiv 2606.02011arxiv.orgCited in the survey line 45, the survey line 47, low-bit numerics line 60, low-bit numerics line 90, low-bit numerics line 91, low-bit numerics line 92
- arXiv 2607.26515arxiv.orgCited in low-bit numerics line 153
- arXiv 2607.26571arxiv.orgCited in the survey line 9, the survey line 29, the design line 25, the design line 197, hardware and serving line 93, hardware and serving line 94
- arXiv 2608.24070arxiv.orgCited in the survey line 47, low-bit numerics line 130, low-bit numerics line 131
- arXivarxiv.orgCited in moe and conditional compute line 135
- arXiv 2609.16338arxiv.orgCited in the survey line 43, the survey line 47, low-bit numerics line 22, low-bit numerics line 23, low-bit numerics line 33, low-bit numerics line 55, low-bit numerics line 56, low-bit numerics line 132, low-bit numerics line 151
- DeepSeek-V4.1-Flasharxiv.orgCited in low-bit numerics line 93
- Cartridges++arxiv.orgCited in the design line 128, the design line 195
- Dual-QKarxiv.orgCited in low-bit numerics line 93
- Decryptdecrypt.coCited in the survey line 71, decoding and test-time compute line 62, decoding and test-time compute line 77, decoding and test-time compute line 168
- DeepLearning.AI The Batchdeeplearning.aiCited in attention and kv cache line 40
- DeepMinddeepmind.googleCited in the survey line 71, decoding and test-time compute line 76
- Red Hatdevelopers.redhat.comCited in the survey line 47, low-bit numerics line 129
- EE Timeseetimes.comCited in the survey line 87, the survey line 89, hardware and serving line 36, hardware and serving line 60, hardware and serving line 61, hardware and serving line 62, hardware and serving line 63, hardware and serving line 64, hardware and serving line 66, hardware and serving line 139
- Emergent Mind summary of arXiv 2502.06703emergentmind.comCited in decoding and test-time compute line 162
- Emergent Mind summary of DeepSeek-R1 paperemergentmind.comCited in distillation and cost trends line 83
- Wikipedia, 1.58-bit LLMen.wikipedia.orgCited in low-bit numerics line 26, low-bit numerics line 32, low-bit numerics line 113
- Epoch AIepoch.aiCited in distillation and cost trends line 29, distillation and cost trends line 30, distillation and cost trends line 84, distillation and cost trends line 145
- Epoch AIepoch.aiCited in distillation and cost trends line 82
- Epoch AIepoch.aiCited in the survey line 25, distillation and cost trends line 38, distillation and cost trends line 39
- Epoch AIepoch.aiCited in the survey line 9, the survey line 23, the survey line 25, decoding and test-time compute line 128, decoding and test-time compute line 167, distillation and cost trends line 10, distillation and cost trends line 11, distillation and cost trends line 12, distillation and cost trends line 13, distillation and cost trends line 14, distillation and cost trends line 15, distillation and cost trends line 147, distillation and cost trends line 150
- FourWeekMBAfourweekmba.comCited in hardware and serving line 33
- Maxim AI on LLaDA2.1getmaxim.aiCited in decoding and test-time compute line 88
- microsoft/BitNetgithub.comCited in the survey line 43, low-bit numerics line 20, low-bit numerics line 21, low-bit numerics line 54
- Graphcore Researchgraphcore-research.github.ioCited in the survey line 55, moe and conditional compute line 131
- HF DeepSeek-V4huggingface.coCited in the survey line 11, the survey line 63, attention and kv cache line 55, attention and kv cache line 137, attention and kv cache line 170, attention and kv cache line 186
- HF/NVIDIAhuggingface.coCited in the survey line 65, attention and kv cache line 28, attention and kv cache line 187
- Hugging Face model cardhuggingface.coCited in moe and conditional compute line 12, moe and conditional compute line 114, moe and conditional compute line 155
- HF Papershuggingface.coCited in the survey line 53, distillation and cost trends line 100, distillation and cost trends line 101, distillation and cost trends line 102, distillation and cost trends line 103, distillation and cost trends line 104, distillation and cost trends line 105, distillation and cost trends line 127
- ICLR 2026 oraliclr.ccCited in attention and kv cache line 37
- Memory Layers at Scaleicml.ccCited in the design line 42, the design line 194
- Inception bloginceptionlabs.aiCited in decoding and test-time compute line 63
- Inception, Introducing Mercuryinceptionlabs.aiCited in decoding and test-time compute line 60
- Introl summaryintrol.comCited in the design line 42
- Kaitchupkaitchup.substack.comCited in attention and kv cache line 17, attention and kv cache line 138
- Let's Data Scienceletsdatascience.comCited in the survey line 71, decoding and test-time compute line 70
- LMSYSlmsys.orgCited in the survey line 33, the survey line 53, moe and conditional compute line 66
- LMSYS blog, Dec 1 2025lmsys.orgCited in decoding and test-time compute line 17
- LMSYSlmsys.orgCited in moe and conditional compute line 91
- LMSYSlmsys.orgCited in the survey line 71, decoding and test-time compute line 83
- Apple ML Researchmachinelearning.apple.comCited in distillation and cost trends line 106
- Raschka, "A Dream of Spring for Open-Weight LLMs" (Jan-Feb 2026 architectures)magazine.sebastianraschka.comCited in attention and kv cache line 15, attention and kv cache line 16, attention and kv cache line 56, attention and kv cache line 57, attention and kv cache line 58, attention and kv cache line 108, attention and kv cache line 169
- Marginal Revolution summarymarginalrevolution.comCited in distillation and cost trends line 13
- MiniMaxminimax.ioCited in the survey line 65, attention and kv cache line 108, attention and kv cache line 198
- NeurIPS 2026neurips.ccCited in moe and conditional compute line 164
- SemiAnalysisnewsletter.semianalysis.comCited in distillation and cost trends line 128, distillation and cost trends line 149
- SemiAnalysisnewsletter.semianalysis.comCited in the survey line 7, the survey line 35, the survey line 81, the survey line 85, hardware and serving line 14, hardware and serving line 15, hardware and serving line 16, hardware and serving line 17, hardware and serving line 18, hardware and serving line 21, hardware and serving line 39, hardware and serving line 96, hardware and serving line 120, hardware and serving line 121, hardware and serving line 138
- Photonics Onlinephotonicsonline.comCited in hardware and serving line 73
- PrismMLprismml.comCited in the survey line 43, low-bit numerics line 24, low-bit numerics line 59
- ICLR 2025proceedings.iclr.ccCited in distillation and cost trends line 100
- PMLRproceedings.mlr.pressCited in moe and conditional compute line 35
- NeurIPS 2025proceedings.neurips.ccCited in hardware and serving line 70
- Pulsepulse.botCited in the survey line 87, hardware and serving line 37
- NVIDIA Researchresearch.nvidia.comCited in the survey line 41, the survey line 47, low-bit numerics line 16, low-bit numerics line 17, low-bit numerics line 64, low-bit numerics line 65, low-bit numerics line 87, low-bit numerics line 88, low-bit numerics line 89, low-bit numerics line 112
- NYU Shanghai RITSrits.shanghai.nyu.eduCited in low-bit numerics line 69, low-bit numerics line 152
- AMD ROCm blogrocm.blogs.amd.comCited in low-bit numerics line 150
- Semiconductor Engineeringsemiengineering.comCited in the survey line 89, hardware and serving line 70, hardware and serving line 140
- Shattered.ioshattered.ioCited in hardware and serving line 33
- SiliconANGLEsiliconangle.comCited in the survey line 89, hardware and serving line 65, hardware and serving line 66, hardware and serving line 95
- Silicon Reportsiliconreport.comCited in the survey line 7, the survey line 11, the survey line 69, the survey line 85, the survey line 87, the design line 23, the design line 196, hardware and serving line 12, hardware and serving line 13, hardware and serving line 22, hardware and serving line 23, hardware and serving line 24, hardware and serving line 25, hardware and serving line 26, hardware and serving line 30, hardware and serving line 31, hardware and serving line 39, hardware and serving line 118, hardware and serving line 119, hardware and serving line 122, hardware and serving line 141
- IEEE Spectrumspectrum.ieee.orgCited in the survey line 89, hardware and serving line 69
- Spheron, "MoE cost per token: why active parameters set your GPU bill"spheron.networkCited in moe and conditional compute line 80
- Spheron on vLLM NVFP4 KVspheron.networkCited in low-bit numerics line 93
- StorageReviewstoragereview.comCited in the survey line 81, hardware and serving line 29, hardware and serving line 32, hardware and serving line 117, hardware and serving line 122
- The Decoderthe-decoder.comCited in the survey line 63, attention and kv cache line 40, attention and kv cache line 157, attention and kv cache line 167
- The Quantum Insiderthequantuminsider.comCited in the survey line 89, hardware and serving line 73, hardware and serving line 142
- tianpan.co, "MoE serving production quirks" (Apr 2026)tianpan.coCited in moe and conditional compute line 80
- Together AI Mamba-3 blogtogether.aiCited in attention and kv cache line 37, attention and kv cache line 93
- Tom's Hardwaretomshardware.comCited in the survey line 85, low-bit numerics line 148
- Tsinghua Universitytsinghua.edu.cnCited in distillation and cost trends line 61
- VentureBeatventurebeat.comCited in the survey line 51, moe and conditional compute line 23, moe and conditional compute line 77, moe and conditional compute line 167
- VentureBeatventurebeat.comCited in attention and kv cache line 157
- victorgrosser.nlvictorgrosser.nlCited in the survey line 51, moe and conditional compute line 21
- vLLMvllm.aiCited in the survey line 47, the survey line 65, attention and kv cache line 136, attention and kv cache line 197
- vLLM DeepSeek-V4 blogvllm.aiCited in attention and kv cache line 55