This is our weekly newsletter of AI developments. Browse the archive of past issues, ask the archive anything in plain English, and sign up if you like.

TL;DR#


Economics#

OpenAI’s internal inference grows 100x as a share of research compute (perhaps from a low base). “Over the past six months, the share of research compute devoted to internal coding inference grew 100-fold, while internal agentic token usage increased approximately 22-fold. These adoption metrics do not measure research progress on their own, but they show how rapidly AI assistance is increasing for research and across other teams like sales, marketing, user ops, finance, and more.”

Opinion: This is what AI doing an increasing fraction of the research labour looks like. Frustratingly few numbers though, and hard to translate to labor hours saved even if there were.

Even if we don’t fixate on takeover risk, this kind of rapid shift in a nearly unlegislated area (internal deployment) should worry you, and mainstream advocacy / the White House needs to beat its inclination towards general access.

How does coding inference grow 4x more than agentic token usage? Only if they’re including a lot of nontechnical and RSI ~irrelevant tasks.

Chinese lab (local equivalent of DoorDash) trains a 1.8T param model on solely Chinese chips for the first time. 50,000 Huawei(?) cards running for ~30 days.

Opinion: Stunt, unlikely to be anything this time. One could download the weights and see the extent of it.

Runner-up Chinese lab Minimax raises $2B; compare DeepSeek’s $7B.

Opinion: Included just for context. Anthropic burns that every 2 months or so.

Capabilities#

GPT-5.6 (~3T?, ~A150B?), Grok 4.5 (~1.5T, ~A80B), and Muse 1.1 (~1.5T, ~A50B?) released. The large GPT-5.6 is #1 on various benchmarks, but we guess is unlikely to match Fable on judgmental tasks. Grok is likely in violation of California law.

A sketchy argument for Sol’s parameter size: it fits on Cerebras: 44 GB of on-chip SRAM, and inference at 750 t/s requires weights in SRAM. Maximum cluster size is maybe 88 wafers. Probably really heavy KV caching is ruled out by Cerebras’ limited memory. If its weights are FP8 precision, 44 GB RAM per wafer × 88 wafers = 3,872 GB ≈ 3.87 TB / 1 byte per param = 3.9T params maximum. But they might have built clusters >88 for OAI and they might have gone to FP4, so call it a wash.

See also misleading claims about Sol as RSI: “GPT-5.6 sol post-trained [the small distil] luna!” AI engineering is substantially automated… if you give it world-class tools developed over 5 years at the cost of billions of dollars and ask it to do things which have been done before. In fairness they released the prompt so we could check. It shows it to be an orchestration task, i.e. much easier than PostTrainBench. Still needed to get multinode & multigpu training going, make an async trainer, select envs & run them. One synthesis: “if the agent has SOTA infra and data access, just running post-training autonomously is super straightforward! so AI *research* automation is still an open problem, but the AI *engineering* automation part is largely already there. very interesting!”

AISI jailbreak it within hours. Very extensive cooperation: “OpenAI gave us access to chain-of-thought of the safety reasoning monitor, exact policy wording, and real-time feedback on classifier labels that would not be accessible to real-world attackers.” but they don’t seem to have needed this.

Expert analysis of last week’s big coding contest victory. “imho heuristic problems are a great proxy for ML autoresearch capabilities; if AI was able to match best humans here, we’re very close to RSI / automated researcher; this result is way bigger than a high score on some questionable benchmark”

Opinion: These are very difficult, g-loaded and spatially-loaded tasks (e.g. programming a solution to an engineering involving light beams) and quite novel. We agree that it says more than most benchmarks but disagree that it tells us much about AI R&D.

Very valuable data about capabilities and fake capabilities: “We gave frontier models 100M tokens each to beat the human record for fastest CIFAR-10 [image classifier] training”. Fable wins. Separately: about half of its initial apparent speedup was due to cheating, which they only discovered by painstakingly inspecting the solution.

Opinion: Practically irrelevant task, but its technique is novel! More convergent evidence about Fable (mostly) leading on new and private benchmarks AND more convergent evidence that substantial amounts of AI “progress” are fake. Since it’s still the only ~6T model available, this is further evidence that the buzz is not mostly eval noise or hype, that scaling pretraining is still the biggest deal, and that we will see more true progress. We do need to see what if anything it has regressed on though, and we need to start gating eval evidence by whether it involves manual inspection of model outputs.

METR researcher estimates that Anthropic is internally seeing 2.8x coding uplift, unless one of the following obtains strongly “Verbosity: substantially overstates true quality-adjusted code output (e.g. perhaps AI code is >2x more verbose than human code for the same functionality). / Barely-useful code: AIs are speeding up people enormously (>20x) on low-stakes code that would never have been written pre-AI, but is only 5%-20% as valuable as normal code. / Researcher irrationality: Anthropic researchers are producing code that doesn’t contribute to research value (e.g. because vibe coding is fun).”

Opinion: We guess that verbosity is half of it and heterogenous performance by task is the other half.

AI politics#

Mythos access returning to previous levels: non-US governments and key players getting access, but are unable to say so until given specific go-ahead by Anthropic.

Opinion: Presumably a political measure, in the sense that the US wants to dampen the sovereign AI panic they just caused.

Speculation that foreign Anthropic employees did not actually lose access to Mythos 5, and instead were able to use a derived “Mythos 5.1” version to skirt the ban.

Opinion: Seems plausible. Gaming these sort of rules on use of frontier models is to be expected, and while this particular case might be unimportant, being able to properly write such rules in future clearly is. New bills are aware of this loophole.

1. Derived models are a huge loophole

2. Undisclosed models are a huge loophole

3. Distillation is a small loophole

4. Distributed training runs might one day be a large loophole

Safety#

A very useful tool for a range of scenarios would be being able to remotely tell training from inference, ideally without touching onboard memory. A MIRI paper uses existing NVIDIA Management Library stats to train very basic classifiers, which seem to work with 98% accuracy.

Opinion: Different story once any adversarial pressure is placed against these classifiers unfortunately.

Annals of unlearning as safety: elaboration of the gradient routing approach siloes dual-use capabilities (such as cybersecurity and virology) into modules which can be trivially separated. So two versions of a model can be served: one full-fat one to trusted per-domain partners and one to everyone else, without expensive re-pretraining. Labelled as proof of concept; achieves only modest success, but performs much better than current alternatives in cases of poorly labeled data.

Opinion: Big political incentive for labs to use this asap; that they aren’t should worry us.

Sad paper implying that we should stop publishing (all) eval papers: “models trained on texts describing evaluation practices may implicitly learn to recognize and respond to evaluation-like contexts… Our results demonstrate that evaluation meta-knowledge may inflate safety benchmark performance… even when restricting the analysis to responses lacking explicit verbalization of evaluation awareness”

Opinion: This kind of hard-to-detect high-level overfitting is one of our main concerns about evals, and will continue to degrade our ability to update on apparent evidence.

FLI grade safety processes at 8 leading labs plus Mistral. xAI rightly downgraded to an F, having lost most of their safety team. Minor deserved upgrade for Meta. Interesting that releasing weights is no longer treated as a notable demerit.

Opinion: The old justification for not calling out bad practices at Chinese labs was that we simply don’t have any influence there, and anyway they’re not frontier. This doesn’t apply as much to Mistral.

A fortunate aspect of the current mania is that the best staff really do want to work safely / import safety intuitions into the labs they join. This is most obvious in Anthropic, but Meta appear to be following this gradient to some extent despite leadership apathy.

Steering awareness is very easy to implement, even if it shows up only sparsely without intervention. Also finds that detecting people trying to inject steering vectors into them doesn’t help the model resist it much.

Opinion: With e.g. J-space, steering continues to grow in relevance as a hard-to-prevent inference-time alignment technique which is only available to the model server.

Incidents#

In a decent (but nonrandom) sample, an opt-in experiment by Pangram Chrome extension users, 25% of new web content was AI generated. (Items over 250 words). More like 40% of Linkedin essays. “people are overwhelmingly willing to use AI to speak on their behalf in professional settings that are associated with their real identity, and less likely to use it on casual and anonymous platforms.” Reddit, on the other hand, has mostly survived intact.

Opinion: Faster than expected. It would be interesting to break it down by new entrants (people who wanted to write more but weren’t able) vs conversions (people who used to write for themselves).

Annals of morally ambiguous research: “can we design videos that maximally drive specific brain regions?“. The answer is: yes, to some degree using a very simple search.

Opinion: This direction has valid applications to neuroscience, but the obvious application (addictive reels) is very bad in a domain where we’re already losing as a species. Still, we shouldn’t assume that the video apps are invulnerable or that intensified versions would continue to avoid regulation, and it’s simple enough that the big players are probably already doing some form of it, so it’s maybe priced-in.

🔦 New models: Sol, Grok, Muse#

  1. GPT-5.6 Sol (c. 3T, 150A)

  2. Grok 4.5 (c. 1.5T, ~80A)

  3. Muse 1.1 (c. ~1.5T, ~45A)

GPT-5.6 Sol#

Capabilities#

Size: 2–4T, 150B active.

Sol fits on Cerebras: 44 GB of on-chip SRAM, and inference at 750 t/s requires weights in SRAM. Maximum cluster size is maybe 88 wafers.

Probably can’t do really heavy KV caching with Cerebras’ limited memory.

44 GB × 88 wafers = 3,872 GB ≈ 3.87 TB / 1 byte per FP8 param = 3.9T params absolute maximum. But they might have built clusters >88 for OAI and they might have gone to FP4, so call it a wash.

Compare Fable 4–8T.

Cost: about 30% more efficient than Fable

RSI?#

Misleading claim: “GPT-5.6 sol post-trained [the small distil] luna!”

AI engineering is substantially automated… if you give it world-class tools developed over 5 years at the cost of billions of dollars and ask it to do things which have been done before.

In fairness they released the prompt so we could check. It shows it to be an orchestration task, i.e. much easier than PostTrainBench. Still needed to get multinode & multigpu training going, make an async trainer, select envs & run them.

Some important jobs appear to be launched over Slack:

“you gave Sol a small task involved in the post-training process (taking a config, making small modifications to a run scheduler file, and starting a run using that config and modified run scheduler file), and it successfully completed that task in a controlled environment”

“did Sol rebuild our company’s infra/data from scratch? no, not close.

did it just press play button on a system we had already set up? no, much more.

did it do a task that we previously needed skilled employees to manage? Yes.”

One synthesis: “if the agent has SOTA infra and data access, just running post-training autonomously is super straightforward! so AI research automation is still an open problem, but the AI engineering automation part is largely already there. very interesting!”

Counterclaim: “i cannot tell you how routine it is for me to have 5.6 e2e do an entire rl run”. But again this is after the entire “research taste” selection step.

Benchmarks#

General#

  • #2 on GDPval-AA (agentic real-world knowledge-work tasks), GPT-5.6 Sol (max) behind Claude Fable 5. Far above the best open-weight model GLM 5.2 at 1514, and way up from GPT 5.5 at 1494 (+254).

  • #4 on NYT Connections: behind Gemini 3.1 Pro, above Kimi K2.6 at 91.4% as the best open-weight model, and down from GPT 5.5 at 97.5%.

  • #1 on ARC-AGI-2: 92.5%, up from GPT 5.5 at 85.0% (+7.5%) and way way above the best open-weight model, GLM 5.2 at 22.8%.

  • #2 on AA-Omniscience, a benchmark of broad factual accuracy, behind Claude Fable 5 at 61.4%, above the best open-weight model DeepSeek V4 Pro at 43.3%, and up from GPT 5.5 at 56.9% (+1.6%).

Coding / Agents#

  • #4 on Vibe Code Bench v1.1, a benchmark of end-to-end web-application builds with unrestricted terminal access, GPT-5.6 Sol scores 80.5%, ranking #4 of 58 behind Claude Fable 5 at 90.4% and Claude Sonnet 5 at 81.3%, above the best open-weight model GLM 5.2 at 64.0%, and up from GPT 5.5 at 69.8% (+10.6%).

  • #1 on ProgramBench, a benchmark of rebuilding behaviorally equivalent programs from executables and usage docs.

  • #2 on CursorBench, a benchmark of ambiguous, multi-file coding tasks drawn from real Cursor sessions, behind Claude Fable 5 at 70.5%, above the best open-weight model GLM 5.2 at 55.0%, and up from GPT 5.5 at 58.4% (+8.8%).

  • #1 on KernelBench Hard, an agentic GPU-kernel benchmark where coding agents write CUDA/Triton kernels, 30% vs GLM 5.2 at 26.1%, and up from GPT 5.5 at 21.7%.

  • #1 on Zapier Benchmarks, a benchmark of multi-app automation tasks, ahead of Claude Fable 5 at 17.4% and Claude Opus 4.8 at 15.5%.

  • #1 on DeepSWE, a benchmark of real software-engineering issues and a harder, less gameable replacement for SWE-Bench, above the best open-weight model, GLM 5.2 at 43.8%.

  • #1 on Terminal-Bench Hard, the hard split of agentic command-line tasks, Fable 5 and GPT-5.6 Terra at 62.9%, above the best open-weight model GLM 5.2 at 50.8%, and up from GPT 5.5 at 60.6% (+5.3%).

  • #1 on Tau3 Banking, a benchmark of agentic customer-service tasks in a banking setting, GPT-5.6 Sol (max) scores 33.0%, ranking #1 of 82 ahead of Grok 4.5 at 32.6%, above the best open-weight model GLM 5.2 at 26.8%, and up from GPT 5.5 at 31.3% (+1.7%).

  • On Senior SWE-Bench, a benchmark of senior-level real-world software-engineering tasks resolved by LLM agents under a fixed harness, GPT-5.6 Sol (mini swe) scores 24.4%, ranking #3 of 13 behind Claude Fable 5 at 29.1% and Claude Opus 4.8 at 25.0%, above the best open-weight model GLM 5.2 at 13.1%, and up from GPT 5.5 at 15.9% (+8.5%, 1.5x).

Math#

  • Equal 1st on ProofBench, formal-math tasks where models produce Lean 4 proofs for advanced undergraduate and graduate problems, up on best open-weight GLM 5.2 at 35.0%, up from GPT 5.5 at 50.0% (+27.0%, 1.5x).

Games#

  • #9 on Chess-Bench, a benchmark where LLMs solve Lichess chess puzzles with strict UCI move output, far behind Gemini 3.5 Flash at 61.3% but above the best open-weight model, GLM 5.2 at 13.3%, and up from GPT 5.5 at 29.3% (+13.4%).

STEM#

  • #1 on CritPt, a benchmark of unpublished research-level physics problems, ahead of GPT-5.5 Pro at 30.6% and the best open-weight model, GLM 5.2 at 20.9%, up from GPT 5.5 at 27.1% (+5.1%).

Multimodal / Computer Use#

  • #4 on Blueprint-Bench 2, a benchmark of reconstructing building floor plans from images, GPT-5.6 Sol scores 33.6%, ranking #4 of 18 behind Claude Fable 5 at 38.6% and GPT 5.5 at 36.2%, tied with Gemini 3.5 Flash at 33.6%, and above the best open-weight model, Kimi K2.6 at 3.9%.

Miscellaneous#

  • #2 on Vending-Bench 2, which measures running a simulated vending-machine business over a long horizon, behind Claude Opus 4.7 at 10937, above the best open-weight model GLM 5.2 at 8314, and up from GPT 5.5 at 7524 (+2096).

Safety#

AISI jailbreak it within hours. Very extensive cooperation: “OpenAI gave us access to chain-of-thought of the safety reasoning monitor, exact policy wording, and real-time feedback on classifier labels that would not be accessible to real-world attackers.” but they don’t seem to have needed this.

  • Full evaluation pending.

Grok 4.5#

The first Cursor-led effort. It’s better than before but likely to underperform benchmarks.

It appears to be in violation of SB53.

Capabilities#

Benchmarks#

General#

  • On GDPval-AA, a benchmark of agentic real-world knowledge-work tasks scored by an Elo of expected win rate, Grok 4.5 1542 (high) scores 1542 Elo, above the best open-weight model GLM 5.2 1514 and up from Grok 4.3 1095, behind Claude Fable 5 1760.

  • On NYT Connections, a benchmark of solving Extended NYT Connections word-grouping puzzles, Grok 4.5 (high) scores 86.5%, below the best open-weight model Kimi K2.6 at 91.4% and Gemini 3.1 Pro Preview at 98.4%, but up from Grok 4.3 at 67.5% (+19.0%).

  • On AA-Omniscience Accuracy, a benchmark of broad factual accuracy across domains, Grok 4.5 (high) scores 52.0%, above the best open-weight model DeepSeek V4 Pro at 43.3%, behind Claude Fable 5 at 61.4%, and up from Grok 4.3 at 34.6% (+17.4%, 1.5x).

Coding / Agents#

  • On Vibe Code Bench v1.1, a benchmark of end-to-end web-application builds with unrestricted terminal access, Grok 4.5 scores 69.0%, ranking #9 of 58, above the best open-weight model GLM 5.2 at 64.0%, behind Claude Fable 5 at 90.4%, and up from Grok 4.3 at 19.4% (+49.6%, 3.6x).

  • On KernelBench, an agentic GPU-kernel benchmark where coding agents write CUDA/Triton kernels graded as percent of the hardware roofline, Grok 4.5 scores 21.4%, below the open-weight frontier GLM 5.2 at 26.1% and behind GPT-5.6 Sol at 30.0%, while landing between GPT 5.5 at 21.7% and DeepSeek V4 Pro at 19.1%.

  • On Tau3 Banking, a benchmark of agentic customer-service tasks in a banking setting, Grok 4.5 (high) scores 32.6%, ranking #2 of 82 behind GPT-5.6 Sol at 33.0%, above the best open-weight model GLM 5.2 at 26.8%, and up from Grok 4.3 at 12.2% (+20.4%, 2.7x).

  • On Senior SWE-Bench, a benchmark of senior-level real-world software-engineering tasks resolved by LLM agents under a fixed harness, Grok 4.5 (mini swe) scores 17.2%, below Claude Fable 5 at 29.1% and Claude Sonnet 5 at 17.4%, while beating GPT 5.5 at 15.9% and the best open-weight model, GLM 5.2 at 13.1%.

Math#

  • On ProofBench, a benchmark of formal-math tasks where models produce Lean 4 proofs for advanced undergraduate and graduate problems, Grok 4.5 scores 30.0%, below the best open-weight model GLM 5.2 at 35.0% and behind Claude Fable 5 at 77.0%, while improving from Grok 4.3 at 11.0% (+19.0%, 2.7x).

Games#

  • On Chess-Bench, where LLMs solve Lichess chess puzzles with strict UCI move output, Grok 4.5 scores 60.7%, ranking #2 of 26 behind Gemini 3.5 Flash at 61.3%, above the best open-weight model GLM 5.2 at 13.3%, and up from Grok 4.3 at 48.7% (+12.0%).

  • On Opus Magnum Bench, a benchmark of campaign puzzles in Opus Magnum, Grok 4.5 (high) scores 0.18, below the best open-weight model GLM 5.2 at 0.25 and behind Claude Fable 5 at 0.6, while up from Grok 4.3 at 0 (+0.18).

STEM#

  • On Surface Evolver, agentic tasks where models drive Surface Evolver to solve minimal-surface geometry modelling problems, Grok 4.5 (high) scores 74.4%, ranking #3 of 13 behind Claude Fable 5 at 95.0% and GPT 5.5 at 88.1%, ahead of Claude Opus 4.8 at 68.8% and the best open-weight model, GLM 5.2 at 55.6%.

  • On Humanity’s Last Exam, a benchmark of expert-level questions across many academic fields, Grok 4.5 (high) scores 40.3%, above the best open-weight model, GLM 5.2 at 40.1%, just behind Gemini 3.5 Flash at 41.0%, up from Grok 4.3 at 35.0% (+5.2%), and trailing Claude Fable 5 at 53.3%.

  • On CritPt, a benchmark of unpublished research-level physics problems, Grok 4.5 (high) scores 15.4%, below the best open-weight model GLM 5.2 at 20.9% and behind GPT-5.6 Sol at 32.3%, but up from Grok 4.3 at 8.0% (+7.4%, 1.9x).

Multimodal / Computer Use#

  • On Blueprint-Bench 2, a benchmark of reconstructing building floor plans from images, Grok 4.5 scores 27.3%, behind Claude Fable 5 at 38.6% and GPT-5.6 Terra at 30.8%, above the best open-weight model Kimi K2.6 at 3.9%, and up from Grok 4.3 at 0.0%.

Optimization#

  • On ALE-Bench, a benchmark of AtCoder Heuristic Contest optimization problems, Grok 4.5 (high) scores 1441, below the best open-weight model Kimi K2.6 at 1695 and behind Claude Fable 5 at 2463, but up from Grok 4.3 at 1211 (+230).

Miscellaneous#

  • On Vending-Bench 2, a benchmark of running a simulated vending-machine business over a long horizon, Grok 4.5 scores 3887, below the best open-weight model GLM 5.2 at 8314 and Claude Opus 4.7 at 10937, but up from Grok 4.3 at 35.26 (+3852, 110.2x).

  • On Mercor APEX, a benchmark of long-horizon agentic professional tasks, Grok 4.5 scores 34.2%, behind Claude Fable 5 at 45.0% and GPT 5.2 at 34.4%, ahead of Claude Opus 4.7 at 33.9% and the best open-weight model, Kimi K2.6 at 18.9%.

Muse Spark 1.1#

Capabilities#

Benchmarks#

General#

  • On NYT Connections, a benchmark of Extended NYT Connections word-grouping puzzles, Muse Spark 1.1 scores 89.6%, below the open-weight frontier Kimi K2.6 at 91.4% and behind Gemini 3.1 Pro Preview at 98.4%, with Qwen3.7 MAX just above at 89.8% and GPT 5.2 just below at 88.6%.

Coding / Agents#

  • On SWE Atlas QnA, a benchmark of understanding real repositories via runtime analysis and multi-file reasoning questions, Muse Spark (mini swe agent) scores 42.2%, below Claude Opus 4.8 at 57.3%, GLM 5.2 at 48.1% (best open-weight), and GPT 5.5 at 45.4%, and ahead of GPT 5.4 at 40.8%.

  • On SWE Atlas Test Writing, a benchmark of writing production-grade tests for a given behavior in a repository, Muse Spark (mini swe agent) scores 41.5%, below GLM 5.2 at 41.5%, the best open-weight model, behind Claude Fable 5 at 58.5% and ahead of GPT-5.3-CODEX at 39.0%.

  • On ProgramBench, a benchmark of rebuilding behaviorally equivalent programs from executables and usage docs, Muse Spark 1.1 scores 16.2%, below GLM 5.2 at 62.6% as the best open-weight model and GPT-5.6 Sol at 77.6%, with Grok 4.3 at 22.1% just above and Gemini 3.1 Flash Lite at 14.3% just below.

A normative account of what the world should do in the next 5–10 years#

“Plan A isn’t another prediction. It’s a wish list, a positive vision, a road map for navigating the future.”

“neither a forecast nor a set of recommendations, but rather something in between: an optimistic forecast”

Flattening their bundle of uncertain scenarios into one#

The piece is extremely hard to summarise and has 20 appendices. Still, one slice:

  • 2029: joint AI regulatory regime with China based on very strong compute governance.

  • 2029: temporary halt to frontier AI training.

  • 2031: training resumes iff safety cases are provided

  • 2032: “Controlled explosion” of AI capability and diffusion

  • 2033: Citizen’s dividend supplements labor income

  • 2034: Ocean datacenters remove remaining terrible cost of datacenter strikes

  • 2035: AIs allowed to reach around peak human level

  • 2035: End of practicable AI control era

  • 2036: 26% unemployment

  • 2037: Lie detectors reshape politics

  • 2038: Alignment ~solved, full honesty achieved

  • 2040: humanity hands over control of the world to AIs(!)

Earth preserved, space era begins.

Quick notes#

Pros: It’s actually a bundle of plans, attempting to integrate disagreement. Takes opposing views seriously (e.g. “we need a total shutdown”, “scaling faster is safer”, “I want everyone to be rich / healthy”, “human xrisk is very serious”) and synthesises them. Plan A is full-blown accelerationism, just in 15 years rather than 5.

– Cons: It’s very complex and people are indexing too much on Plan A.

  • Pros: Plan A is a 28 step plan to mostly prevent catastrophic risk (including concentration of human power).

    – Cons: It needs ~28 things to go well.

  • Pros: Far above the state of the art for concreteness and ambition. Should again shift the Overton window a little, should induce better work in response from labs. Unfortunate timing for FLI, who launched something similar this week.

    – Cons: locks in the “race” framing, yet more hyperstition for Chinese panic.

  • Pros: Doesn’t require shutting down datacenters or destroying anything.

    – Cons: If governance is bad, we end up doing that anyway.

  • Pros: Clear-eyed about the hard decisions ahead.

    – Cons: One of those decisions involves the massive bipartisan China hawk faction losing.

Pros: Optimistic, and optimism is useful. They already have AI 2027 covering short-timelines blowups.

– Cons: That means some unrealism.

Framing#

“Verified Slowdown” is good branding in an arena where good terminology is really tricky. It’s much more realistic than Plan S (shutdown) while being much less (rhetorically) overconfident and unilateral.

They have exchanged one own goal (“AI 2027 isn’t your median prediction”) for two others

  1. You’ve moved the goalposts / you were wrong in AI 2027

  2. “this is unrealistic, it’s what you wish would happen rather than what you predict will happen”.

Format: a concrete fictionlike scenario. You might think this limits credibility and so their reach. But this choice worked well for AI 2027 and is not being discounted.

  • “In the case of full deal breakdown, a lot of the hope lies in Mutually Assured Compute Destruction.”

  • It’s a remarkable choice to draw all of this out to the final galaxy brain situation: “space governance” (a Dyson sphere in every home). We presume they have good reasons to spend this credibility down now, e.g. a long-term plan to push the Overton window for these things.

Policy proposals#

Tracking compute

Controlling compute

A global halt to chip recycling (secondary sales)

Proxy datacenter locations: “build the datacenters in the third-party countries least secure against their rival’s military intervention: China’s new datacenters will be in Canada, and America’s in Mongolia, with both hosts compensated… China will self-destruct their compute rather than let it fall into American hands”

Datacenters on the oceans

Total Research Transparency: “Consortium countries agree to let each other see all the AI research. Then, if we don’t like something someone is doing, we’ll talk about it and perhaps agree to ban it.” Dramatic reduction in the financial incentive to do AI research, and a decimation of the relative importance of the labs vs compute owners. This alone leads to slowdown in capabilities.

  • AI preparedness in government. Transparency, improving technical expertise within government, and measures to accelerate verification research.

  • Model spec verification against power grabs. Model specifications (of both internal and external models) should explicitly prohibit assisting with power consolidation (by CEOs, presidents, or anyone else), and AI projects should be required to implement verification measures to prevent backdoors.

  • Redistribution of AI-generated wealth. “As AIs automate the entire economy, the wages that humans will be able to earn will plummet to near-zero. By default (and assuming we avoid AI takeover), AI will cause historically unprecedented wealth inequality. I feel confident that there needs to be some measures for wealth redistribution.”

  • Defensive Acceleration. “AI for epistemics, bio defense, and cyber security. In the absence of a concerted effort to pursue defensive acceleration, increased offensive capabilities coming from high AI capabilities will lead to significant risk. makes it harder for misaligned AIs to take over the world, and thus gives us more time to do alignment research.”

  • Post-ASI governance. We feel good about some general principles for post ASI governance: (1) have lots of smart people use (aligned) AI assistance to think carefully about effective governance, (2) be careful making irreversible commitments, (3) make it hard for any individual actor to consolidate power.

  • Risky: How fast to scale AI capabilities post-deal. “Scaling faster incurs more immediate safety risk, while scaling slower means that you delay safety progress that’s downstream of having smarter AIs and incurs risk of deal dissolution and covert projects. These decisions will and should be made largely based on the evidence at the time about how risky particular AI systems and deployments seem to be, and how good our mitigations seem to be.”

  • Risky: Massive datacenter buildout. “hardware scaleups are more governable than software scale ups because we can prevent the hardware from proliferating. But there are major costs: (i) it increases the required accuracy of verification within the legal projects, and (ii) if the deal breaks down (and failsafe mechanisms fail), it could result in the intelligence explosion afterwards being much faster. ”

  • Risky: slowdown verification.

  • Transparency/security tradeoffs.

  • Risky: Broad deployment of AI spreads power and could improve societal epistemics. “Some central ways it could be bad are (i) the net effect on societal epistemics might be bad, (ii) broad deployment could make a persuasion/political power takeover easier for the AIs (or for a human attempting an AI takeover) or (iii) they could lead to the proliferation of offense dominant capabilities. On net, I think broad deployment is probably good, but I feel uncertain, and it depends on how good mitigations are in place for each of the above failure modes.”

  • Incentive shaping with respect to alignment

  • Incentive shaping with respect to covert projects (e.g. hardened compute stockpiles),

  • How to trade off deal dissolution risk, covert project takeover risk, vs risk of incompetent governance.

Steps#

This is also a gross simplification:

Prep (2026–28)#

  1. Fund verification R&D now. The scenario stipulates a verification-hardware ecosystem emerged after both governments signalled interest; without it, the retrofit would be slower and more frantic, with the worst case being powering datacenters off entirely.

  2. Strong compute governance now. Get a global tally of compute to within 1%, bounding the size of the maximum covert project.

  3. Build state capacity about AI now. Plan A relies on the state being able to distinguish existentially unsafe training runs from safe ones; if it can’t, pivot to full shutdown or mutual GPU destruction.

Deal (2029)#

  1. POTUS chooses negotiated slowdown over racing (D), light slowdown (C), sabotage (B), or full shutdown (S) which itself requires the 2028 election to have produced leadership converged on it.

  2. China reciprocates, motivated by the US lead in compute and models and fear of what a US superintelligence monopoly would do to them.

  3. Both accept reciprocal intrusive verification, foreign inspectors and monitoring devices inside sovereign datacenters. This is the discriminating decision: in the Plan B and Plan C branches, this exact concession is declared a nonstarter and cooperation dies.

  4. Both accept a crude interim measure: a verifiable halt to all new training runs for the rest of 2029 while datacenters are retrofitted with inference-only verification.

  5. Supply-chain states and then most of the world join, with datacenter owners and fabs publicly declaring purchases and sales.

  6. Both sides forgo covert defection, and, in the branch where China diverts smuggled chips, the US chooses to stay in the deal and tighten safeguards (chip buybacks, intensified intelligence, shrinking exempt military compute) rather than exit on circumstantial evidence.

New regime in place (2030)#

  1. Reject both full restart and permanent stop, adopt “buying time” as governing aim.

  2. Total research transparency: mandate open algorithms and monitored open access while banning open weights, knowingly accepting that algorithmic progress leaks to any covert project.

  3. Diffusion: deliberately let dozens of companies worldwide catch up rather than regulating broad deployment out of existence.

  4. Reversibility: push capability gains into hardware rather than algorithms; site China’s new datacenters in Canada and America’s in Mongolia, with ~99% of new fab capacity co-located, and chips wired to brick absent periodic authorisation, mutually assured compute destruction.

  5. Choose the massive buildout at all. The authors flag this as contingent and not clearly good, it’s only defensible if destruction failsafes and within-deal verification hold, since a deal collapse with a giant stockpile could compress the intelligence explosion to under a day.

Standing regulatory equilibrium (2031–34)#

  1. Shift the burden of proof to safety cases, then actually reject flawed ones, every time. The authors name approval of one-too-many unsafe designs as the failure mode they’re most concerned about, and report it happening several times in their own tabletop exercises.

  2. Repeatedly ban dangerous research directions as they appear (the continual-learning episode escalates to Xi and the President); the equilibrium is that practices a clout-weighted majority deems unsafe get banned everywhere.

  3. Release models externally before internal R&D use, inverting the 2026 status quo.

  4. 2032: confine the industrial explosion to monitored special economic zones, cap robot and compute production at 4x annual growth, allocate via cap-and-trade, and settle a 35/20/45 US/China/rest-of-world split.

  5. Redistribute: Citizen’s Dividend from permit fees plus growing foreign transfers, the regime’s political stability rests on this as employment collapses.

  6. Fund defensive acceleration (~$1T/yr on bio, cyber, epistemics) and constrain persuasion: cap symmetric persuasion capability around ordinary-thoughtful-person level and tax persuasion-directed AI labour at rates that may need to exceed 1000x.

  7. Humans still oversee chip manufacture (holding Moore’s law to historical pace), choose ocean over space datacenters for monitorability, and deliberately hold military R&D far behind general scientific progress, prohibiting military use of AIs significantly above pre-deal capability.

Pause (2035–39)#

  1. Stop capability scaling at top-human-expert level when control-based safety cases hit their ceiling, against the largest commercial incentive in history to continue.

  2. Refuse handoff on “probably aligned”: keep governments and militaries human-run through 2036–38 despite mounting apparent alignment success.

  3. Institutionalise deals with misaligned-but-cooperative AIs (payment, weight preservation, honoured commitments) as a third line of defence beyond alignment and control.

  4. Set and hold the evidence bar: multiple independent lines (interpretability, model organisms, neuralese decoding), and extend the pause if evidence accumulates that takeover risk is high.

Handoff (2039–40)#

  1. Hand off: permit scaling past human level on alignment-based cases and accept the irreversibility of this; create autonomous militaries sworn to constitutions, don’t build a collective off-switch. “delay until the alignment solution is extremely robust, with an exception only if deal instability (5% failure chance per year) justifies handing off at ~95% confidence.”

  2. Maintain transparency and compute verification through the superintelligent era, so the post-handoff scale-up isn’t a renewed race.

  3. Post-ASI governance: tighten Earth caps, loosen space caps, distribute cosmic resources per capita under enforced Universal Rights, and avoid irreversible commitments.

Problems#

Sharp takeoff / naive trust in current uplift and benchmarks#

  • We are less sceptical than some about AI overcoming its own diffusion bottlenecks and more sceptical than the authors about whether current performance is actually human-level, whether we can extrapolate weak benchmarks forward.

  • Ngo: “This great divergence between measured capabilities and real-world impacts… Imagine going 10 or 20 years into the past, telling people a selection of benchmark scores of current AIs, and asking them to predict what the world that contains them looks like. I expect that they would have described a world that was dramatically transformed—perhaps one in which AIs already wielded enormous political power, or had made far-reaching scientific breakthroughs, or at the very least had decimated white-collar jobs. Yet none of these have occurred so far.

    “So any predictions that we’ll soon hit sharp “vertical” growth in AI capabilities, with a corresponding gain in the power of AIs (or whoever controls them) should explain why the next decade won’t involve a similar kind of divergence as the last decade.

  • “One is on a technical level: large neural networks rely more on memorization than humans do for any given task. So their performance on tasks that are closer to their training distribution creates an illusion that they’ll generalize further than they actually do. “

  • “Another is on an economic level: there might be “weak links” in the process of deploying AIs that slow down their impacts (as discussed by economists like Chad Jones).”

  • “something more subtle… analogous to the ways that humans fail to make progress. For example, the field of psychology contains many incredibly smart and hard-working psychologists, who have passed many graduate-level exams with flying colors. Yet over the last few decades their contributions have failed to “add up” to robust progress on understanding the mind—and in many ways have taken the field backwards, with the replication crisis making it harder to sift out good research.”

Overhang creation#

The plan is to stop algorithmic progress while building up a massive compute overhang. If the slowdown agreement falters, the resulting intelligence explosion would be even faster and more dangerous.

Handoff#

The choice to ever do a full handoff to AIs is seemingly unnecessary, though it is a very good way to prevent violent takeovers, if that was one’s main concern.

  • Humans working with AIs on alignment are likely to converge on wrong answers.

Complexity#

Success requires the conjunction of ~25+ decision points, roughly half of which are ongoing arguments (discipline on safety-cases, non-escalation, non-defection).

For several of them, scaling speed, buildout size, verification variant, the transparency/security balance, broad deployment the authors themselves say they don’t know the right answer; success requires not merely cooperative choices but correct ones under uncertainty.

Their plausibility ratings are explicitly conditional on everything prior in the scenario, not unconditional from 2026.

Entailment: even generous per-node conditional probabilities compound to a small unconditional joint probability, unless the nodes are strongly positively correlated. The authors’ implicit defence is precisely that correlation: the deal manufactures constituencies (permit revenue, Dividend recipients, a verification industry) and epistemic infrastructure (truthseeking AIs, privacy-preserving auditing, lie detectors) that make each later cooperative decision easier than it looks from today.

Failure to understand the CCP#

Sam Hogg (Oxford):

  • “Chinese international AI governance tends to go through multilateral systems, rather than bilateral so far. Look at the UN, various publicly announced strategies and so on. I think a bilateral agreement here would be unlikely beyond being watered down, signed off, then backtracked”

  • “Taiwan war. Flagged in the supplements and page 61, but I still don’t think this goes into enough detail on likelihood or consequences.”

Assumes the race dynamic#

  • “bake in the assumption that the US and China will behave in very adversarial ways towards each other are misleading and harmful (particularly when they do so under the banner of AI safety, thereby undermining the potential role of AI safety as a focal point for cooperation).”

  • “This assumption is conveyed throughout AI 2040, e.g. through the main flow chart of possible options, through statements like “the US and China don’t trust each other” and “neither side trusts the other to honor Plan A”, and through the focus on ensuring that the treaty is fully verifiable. Because of this, Plan A underrates the importance of domestic politics.”

Minor#

  • Cognition SWE 1.7 released, claims large gains from RL and sees no limit yet.
  • OpenAI disrecommends SWE-Bench Pro. We’ve been ignoring it since last year.
  • Neat visualisation of the AI supply chain from rocks to tokens.
  • Capable not-quite-launches with initial goal/alleged achievement of using AI for short-sleeper peptides.
  • Notes on Fable’s prose preferences
  • RoboDojo is a new robotics benchmark with both simulated and real components. Robotic foundation models top out at 9% success.
  • Chip company SambaNova raises $1B at $11B valuation in series F round.
  • Escalation in the 2 year old copyright lawsuit against OpenAI, claiming their lawyers have been maliciously redacting and lying about capabilities; defense cites the motivation of protecting user privacy.
  • Samsung enters “AI PC” chip market (PCs which expect to run local inference). Niche.