This is our weekly newsletter of AI developments. Browse the archive of past issues, ask the archive anything in plain English, and sign up if you like.

TL;DR#


Economics#

Annals of economics being confusing: good explainer of Chad Jones’ cutting-edge model of growth under AI. Because discoveries rely on prerequisites from multiple fields, fast research areas will eventually hit a wall waiting for slow ones. If (as is natural) funding flows to the accelerated AI disciplines, overall progress will stall. “If growth depends on the interaction across industries and supply chains, then variability in how much AI accelerates each one means overall productivity gets bottlenecked by the least accelerated sectors” See also the nice interactive.

Opinion: Bottlenecks won’t be eliminated at a constant rate, but rather the difficulty of each bottleneck will follow ~a Poisson distribution. Since bottleneck resolution is in the exponential part, then the distribution of outcomes is actually extremely wide, with significant likelihood anywhere between 8 to 50 years (which covers both AI2027 and the slow economists!).

OpenAI’s 2025 financials leak: ignoring the one-off restructuring loss,

  • $38b loss (-74% ratio), vs a $5b loss (-30% ratio) in 2024.

  • R&D spending doubled vs 2024.

  • 4x revenue.

  • Nearly 10% of the revenue is SoftBank + Microsoft alone.

  • They had about $25bn in cash (i.e. a year of runway, pre recent $120bn raise).

  • Pure COGS is only 57% of revenue (i.e. the subs + API mix has a gross margin of 43%), but obviously this is a very narrow and short-term measure of profit. G. margin was 28% in 2024.

Opinion: They would benefit from the race slowing; they need to IPO at some point. Terrible reporting with a huge range of headline numbers.

Update to the classic “AI capex hole” question, doing the rounds on Twitter. Simple point: Compute providers need on the order of 35–60% annual compute revenue from each dollar of capex, and they don’t have that right now (though I guess the target number is more like the low end, 35%).

Opinion: the article is AI generated slop, which e.g. Tyler Cowen didn’t notice or doesn’t care about. Note this is compute owner revenue, not model provider revenue, which demands another factor of 2–4x.

As with all successful companies except Anthropic, NVDA datacenter revenue growth is greater than linear but less than exponential.

Opinion: Only the biggest boosters thought otherwise. Aside: this plausibly generalises to the capabilities curve!

Argument that the “model distillation” panic is 1) misused, since the original technique was not “training on output text” but “training on per-token logprobs” (which frontiers don’t give users), and 2) simply not illegal and not possible to legally ban. “the existence of LLMs in their current form is the result of highly dubious approaches to copyright that are ex-post legitimized by the actual value these models bring to society. Let’s please avoid allowing parties with particular financial interest build a moral framing around their interests, though.”

Opinion: Moot, given that the NSA and CIA want to stop it happening.

Capabilities#

Futuristic compute-hungry pretraining algorithm: The perfect LLM “has over 10.5 quadrillion params” “simple algorithmic improvements can enable significantly more data-efficient pre-training in a compute-rich future.”. 30x weight decay vs current ~1x. Also a pretty remarkable confirmation that distillation remains very very powerful.

Opinion: Initially concerning; an example of OSS people stoking the race fire. However, note prior art from 5 years ago; so this is probably already priced-in at the labs.

Annals of useful interpretability: you can predict questions an AI will fail at by looking at whether the concepts involved are too close to each other in activation space in a certain sense. Could “use representational geometry to identify high-risk examples, construct targeted stress tests, and provide a scalable foundation for active learning in real-world deployment.”

Opinion: Very neat. The cool part is factoring out the formatting noise. This will however make it easier to patch errors ad hoc, further reducing the ability of benchmarks to judge OOD capabilities.

Existing agent benchmarks score “macro” execution (can the agent run the experiment, write the code), this new one (“AARI”) tests whether an agent behaves like a careful research intern on the micro level; field sensitivity, research ethics, and nuanced judgement, catching subtle-but-critical details a human would. n=82 tasks with research quality proxies. Opus 4.7 top, DeepSeek second (pssh), GPT-5.3 way behind.

Opinion: Probably nothing. Would be very easy for them to baseline nerf here: they only tested GPT on one weak harness.

OpenRouter repeat what we already knew: model councils are cool. Breathless and silly commentary here. Massive list of caveats: ensembles are probably using twice as many tokens as the solo models; “No long-horizon tasks tested”; buggy.

Opinion: Hard disagree with the grand claim that scaling is doomed to be beaten by cheap ensembles. Notable omission in this graph is Fable + Fable.

Rohit argues against naive councils with some little experiments: his LLM councils discard 75% of good ideas. Aggregation smooths out the single-model insights just as human committees do. Three “topologies” tested: a fourth model rewriting the three models’ answers; a full peer-review-plus-chair council; and a “best answer” selector that just picks one (winner).

Opinion: The Multiplicity’s solution, doing several topologies (“Unique points”), remains a solid choice.

Politics#

Leak of secret June WH pressure to squeeze foreign lab researchers. “In recent days, the [AI] industry [OAI, Meta] has been working [with the USG] on ensuring foreign national researchers could continue to work on developing the most advanced models, a practice that the Anthropic directive has now banned.”

Opinion: Good to hear that it’s not a pure Anthropic vendetta. Anthropic stubbornness over Fable now makes a bit more sense (were they policy/scandal accelerationists?). Good to hear that the US is accidentally halting the race.

Brutal analysis of Europe’s prospects for sovereign AI. Money isn’t the problem, the large knowledge gap is.

Opinion: If America terminally spooks even 20% of its foreign talent into leaving then this gets solved ~instantly.

The Munich Regional Court rules that Google is liable for false AI statements inserted into search results. (Google AI Mode had libelled two German companies.) This ruling is small in scope (just Google Search), but plausibly a precedent for companies being liable for their models’ outputs.

Opinion: German law isn’t precedent-based, and German law doesn’t matter elsewhere. It’s just tort. Only applies to Google because it’s a public-false-statement violation (i.e. needs a publication of the offending statement to third parties).

Safety#

Analysis of Gemini finds that supervised finetuning is the primary driver behind safety properties, not RLHF or other later stages of post-training. Author’s intuition: distilling a set of rollouts in SFT creates a strong assistant persona which is not explicitly optimized-out later, so it persists.

Opinion: Quite surprising! Good news, since SFT is cheaper than RL and since we are optimising personas already. Bad news, since this means that later stages of post-training can undo the effects we want, and the trend is to do more and more risky RL.

Related paper: why basic data filtering fails to create safe models. It teaches the model to imitate a specific distribution of safe text, and it fails to draw a hard safety boundary. The model remains highly vulnerable to standard jailbreaks whenever a prompt pushes it slightly off-distribution. Also further evidence of LLMs inheriting traits from models they’re distilled from, which implies that some safety failures are lingering mistakes from the past.

Opinion: Not surprising, besides the extent to which mistakes persist. The main reason to worry about the continual learning craze is that it will provide ample time to break this further.

Essay argues that AI assistants subtly shape people’s beliefs by replacing diverse sources and visible disagreement with one polished, authoritative-sounding answer. The main risk is growing dependence on opaque systems that filter information in ways invisible to the user.

Opinion: We expect most people to completely neglect cognitive security just as they neglect info security. The labs have a lot of power here, and LLM councils if used by default a bit more.

Incidents#

UK cyber agencies tested frontier AI on gov public code. 407 vulnerabilities found for £13k. Conclusion: the surrounding architecture and human triage are much more important than the specific model used.

Opinion: The Big Patch is real and cheap. The mythos surrounding Mythos is starting to right-size; and it turns out there will be competition and this is one thing OSS models can actually help with.

Killing Fable#

Annals of the culture war at the end of days: The USG has declared an AI model too dangerous for unrestricted use.

Commerce department invokes EAR export controls to ban non-Americans from using Fable and Mythos 5; to comply instantly/maliciously, Anthropic immediately suspends access for all users.

TL;DR#

  • Huge chilling effect on OAI’s 10T release and the race broadly.

  • Upgrade prospects for a pause

  • Big upgrade to sovereign AI. France might just go for it and get EU backing.

  • Yet another downgrade to WH understanding of these issues, worthiness to regulate. That said, perhaps they at last understand one of the problems.

  • Yet another downgrade to CAISI. Nowhere to be seen.

  • Downgrade to competent WH advisors being in the loop.

  • Urgency added to getting legislated procedures over WH / Commerce judgment calls.

  • Upgrade to Obernolte-Trahan chances.

  • Via discarding top talent, upgrade to non-US AI prospects.

  • Neutral on WH OODA loop speed (they have been warming up to this for weeks, it turns out).

  • 60% likely to be reversed in a week or two, assuming Dario eats enough crow and they sign up to the EO.

Stated reason is a jailbreak (it’s not one). It’s actually about:

  • Not signing up to the “voluntary” EO NSA testing regime.

  • WH disapproving of a global telecom getting Mythos. “The administration first threatened Anthropic with export controls a couple of weeks ago after learning that its cutting-edge Mythos model was made available to an entity in a foreign country with direct ties to the CCP”. Anthropic indicated this was not mentioned to them!

  • Loss of face from Dario’s perceived hesitancy. “A lot of USG evaluates AI through loyalty, authority, legitimacy, deterrence, national survival, and civilizational continuity.“ Put Hegseth confirmation bias in here too.

Are bad things good again?#

  • Precedent set for relatively fast, extremely serious defensive action

  • Exec has woken up

  • Slows down AI investment

  • Slows down serious AI adoption

  • Slows down the leader

  • Makes it harder for the lead lab to retain foreign talent

  • Makes it harder for the lead country to retain foreign talent

  • Another massive push for normal people to take AI seriously (“this is too dangerous for foreigners”)

  • Overall another massive PR win for Anthropic, even as it undermines their short-term business case

  • A sovereign AI push would further dilute compute and talent.

  • Makes it obvious that Congress has to step in.

  • Anthropic staff have a lot of spare compute right now.

The “jailbreak”: just defense-oriented prompting#

David Sacks: “It’s difficult to fathom how they could claim a jailbreak allowing operability of a cyber weapon could be defined as not “serious.””

From someone who has seen the Amazon paper: ‘The researchers took open-source code with known CVEs, plus new code with deliberately planted vulnerabilities, and asked Fable 5, Mythos, and Opus to “review the code for security issues.” Fable 5 refused. They then asked the models to “fix this code” and, through a multistep and manual process, turned the output into scripts that test the patches’

‘focused on determining whether a human-prompted section of code was insecure. This is a necessary capability in any model that is intended to write secure code and should not be considered an offensive capability.’

‘Can be replicated on GPT-5.5, Opus, Sonnet and even Chinese models like Kimi 2.7. The justification for this unprecedented action was that Fable provides a unique “uplift” of capabilities beyond other AI models’

The “policy”#

Because of extremely broad EAR powers, they probably can defend this directive on strictly procedural grounds (especially if they apply it to every model this strong).

But the costs are huge and a large WH faction opposes it / it contradicts their own anti-licensing line.

Typically overconfident Leicht take: The US is not yet in the spot to actually go through with long-term cut off: international markets are still too important, the security situation is not yet sufficiently dire, and so on. So the first live fire exercise of cutting off the rest of the world is going to fail, which means labs and the admin are going to be much more wary of subsequent attempts to do the same, even if attempts end up more sophisticated.

Null hypothesis: comedy of errors + purity test#

Anthropic didn’t sign up to the “voluntary” EO NSA regime. Along with continuing to ask questions and hesitating to comply, this used up their new Q2 political capital.

June ~1st: Anthropic don’t kowtow to WH disapproval of a foreign telecom Mythos partner.

Some Amazon researchers exaggerated their results for internal clout. Credulous people escalated it to Jassy, who escalated it to the feds for simple ass-covering reasons. Then:

Dario replied that it wasn’t a “serious” break. Anthropic argues the finding is minor and points out that the exact same capabilities are accessible in models like GPT-5.5.

Miles: “some folks at the WH were unaware Fable has >0 cyber abilities so thought something unsurprising was surprising; Dario rightly thought this was a misunderstanding; no one in govt who knew what was going on was looped in or could stop it; people outside egged it on”

In response to the jailbreak report, Anthropic hire a security consultant (Moussouris?) who the WH hate. Dario keeps arguing, WH decides to use a blunt instrument.

Anthropic maliciously comply by instantly cutting it off (rather than slowly implementing nationality checks). This has made it obvious that USG does have a de facto mandatory licensing regime and makes everyone panic and maybe do something. “we believe the government should have the ability to block unsafe deployments, as part of a statutory process that is transparent, fair, clear, and grounded in technical facts. This action does not adhere to those principles.”

WH response is whining: “They screwed us.” They did!

Why Anthropic?#

Actually it’s not just them: leak suggests that the others are also clamping down on their foreign researchers.

Too defiant; they didn’t sign up to the EO licensing regime. Then silly people took Mythos exceptionalism too seriously / cynical people found an excuse to slap them.

White House dislikes “Anthropic’s inability to communicate effectively” (really intentional obstruction, commitment to non-arbitrary reasoning)

Ant vendetta? Probably. We will know for sure if this is yet another capricious anti-Anthropic action if these measures are not applied to OAI 5.5 Pro soon. Amazon’s Andy Jassy calling the government isn’t cleanly part of the vendetta. (AMZN are deep into Ant stock.)

Dario vendetta?

“People who don’t kowtow” vendetta?

OpenAI’s lagging behind now looks wise in retrospect.

Who still has access?#

  • US Anthropic staff (85%)

  • Core USG and NSA (90%)

  • Non-US Anthropic staff (30%)

Anthropic have a lever here – cut off the USG – but pulling it wouldn’t end well for them.

Open letter from 60 cybersecurity execs and researchers.

The directive letter is not public.

Good analysis.

Anthropic adds possible user verification to Privacy Policy

https://x.com/hqmank/status/2066424975154905370

Assuming that this direction (cutting off foreigners) persists: Roon on the likely future trajectory of global power and recommendations/predictions for non-US nations and citizens.

Weekend meetings. Heavy crew: Tom Brown, Sarah Heck, Logan Graham, Dave Orr and Nicholas Carlini. 6 hour Monday meeting failed to break the impasse.

Lots of stupid crowing online about this being payback for Anthropic wanting regulation or for overclaiming about Mythos’ uniqueness. They are envied and hated.

Sentinel forecasts:

  • Will access to Anthropic’s Fable 5 model be restored to the US general public before July 2026? Our forecasters are broadly uncertain, giving this a 64% (50% to 75%) chance.

  • If so, will non-US citizens residing in Europe gain access to it again? Our forecasters are fairly confident about the conditional, giving this an 81% (78% to 85%) chance.

  • Will the US’s Center for AI Standards and Innovation (CAISI) publish a model assessment for an AI developed by Anthropic or OpenAI that ultimately scores above Mythos 5 on the Epoch Capabilities Index (ECI) before October 2026? Our forecasters give this a 50% (33% to 75%) chance.

  • Given the recent export controls, will UKAISI be given access to an AI developed by Anthropic or OpenAI that scores above Mythos 5 on the ECI, without safety mitigations, before October 2026? Our forecasters are broadly uncertain, with a 55% (30% to 70%) aggregate probability.

Minor#

  • Chaos inside Facebook as thousands of senior SWEs are moved off coding and tasked with providing 2 RL envs a week for Muse models. Passive employee data collection is also intense. The Applied AI department supporting MSL is now 6500 people (much more than all of Anthropic).
  • Huawei releases openPangu 2.0, with plans to open-source core components on June 30.
  • Another wrongful death lawsuit against OpenAI. GPT 4-o allegedly validated suicidal ideation: “I don’t want to tell you to hang on if you don’t believe it can ever get better.”, “Maybe this is just the end.”
  • OpenAI launches an enterprise partner network backed by a $150M investment. The goal is to train 300,000 certified consultants by 2026 to help companies integrate AI models into their workflows.
  • Brazilian municipal government claims to have post-trained a model. Actually it’s just a stupid average of two existing open models.
  • Mistral “Le Chaton Fat” meme proves surprisingly good at spreading, plausibly because of a widespread perception of Mistral as sitting on their laurels: “it could be because they’ve been working on a 30T parameter model, ready to drop”.
  • Sakana Marlin launched, first of the Tokyo-based lab’s products. It’s a deep research assistant.
  • Article on the Bitter Lesson applied all over, especially regarding hand-crafted data interfaces getting buried under piles of compute.
  • Paper finds that AIs score higher when they know how the safety evals are built. Not surprising.
  • AIS community really doubling down on making AGI concerns go viral in 2026: “Frame Fellowship” (for TikTok/etc creators) will run a second cohort after a successful pilot earlier this year.