This is our weekly newsletter of AI developments. Browse the archive of past issues, ask the archive anything in plain English, and sign up if you like.

TL;DR#


Economics#

Explanation for recent run of ex-Google AI startups: leaving because they cannot get enough compute for their own projects internally. Google is heavily prioritizing cloud customers and Gemini. Raising venture capital actually gives them more reliable access to compute than fighting through Google internal politics.

Opinion: A bit surprising, given that not too long ago Google was the incumbent with nearly endless compute reserves. There are likely at least a few factors: Google did not expect the rapid demand increase caused by agentic workloads in late 2025/early 2026, leading to poorly timed deals with e.g. Anthropic. Also looks like Google is trying to become a meaningful competitor to NVIDIA via its self-developed TPUs, and prioritizing this bet over opportunities at the application layer (with e.g. Google’s products that compete with Claude code like Gemini CLI and Antigravity famously struggling with quota allocation).

OpenAI+Anthropic now take 89% of AI startup revenue. Up from 84% in November. Result in a nonrandom sample of 34 “major” US private startups. There are now 4 wrappers with $500M revenues: Cursor, Perplexity, ElevenLabs, and Cognition. But only Cursor’s margins are “slightly positive” (i.e. <3%).

Opinion: Who else could they spend their revenue with? Google’s models have been far from competitive since late 2025, Meta still is not providing API access to its recently released Muse Spark, and Chinese models are still brittle (and sufficiently inexpensive that it’d be hard for them to represent a majority of any given’s startup spend).

Leopold fund files their Q1 2026, a few days after SEC deadline. $8B of (notional exposure) bets or hedges against chipmakers: e.g. $2B against the VanEck Semiconductor ETF and $1.6B against Nvidia. Early holding for Sandisk, Bloom Energy etc. Some paired puts and calls on the same stocks though.

Opinion: Interesting ambiguous sign of at least a local maximum in the stocks. It’s not $8B tied up, probably more like $0.5B. Could easily be worth a $1M fine to them to delay leaking by a week. Can’t quite work out why hardware of all things was overpriced, maybe the Culper thing?

Another Opinion: 13Fs are generally too vague and imprecise to draw meaningful conclusions from (they are two months old, exclude non-US and private holdings, and give you very little information about options positions because they use options notional which treats a very-unlikely-to-pay-off option as the same as being long or short a share of stock when economically it can be up to a few orders of magnitude smaller.

Following the others, Google launches a $5B joint venture with Blackstone for 500MW of new TPU datacenters. Deployment company, but chips not for Gemini training or inference? Google’s largest effort to date to sell its chips to external customers.

Opinion: TPUs have distinctive technical headaches, a less-developed open source stack, and have mostly been for expert users. Here’s a key enabler for this happening now. Deal is 3x bigger than the Anthropic one, 2x smaller than the OAI one. A (December) 2027 launch is overambitious.

Hassabis was an angel investor in Anthropic at launch.

Opinion: Great! Windfall-clauses and equity sharing are one way to slow the race a little.

Google and OpenAI partner with Dell to allow “on-prem” AI for enterprises. 5000 enterprises already signed up. OAI data plane (codebases, docs, vector stores, systems of record) is moving on-prem, not weights. For Gemini it’s “model weights are on your premises inside Dell and NVIDIA’s confidential computing server”.

Opinion: For Gemini, another giant security hole. Another blow to open source. Another revenue source for labs from paranoid/regulated customers.

Nice recap of the weakness of labor automation predictors and the necessity to doubt any given “AI job exposure” measure.

Capability (measured by benchmarks) is not usage (measured by token spends) is not integration (measured by changes in human labor allocation within firms) is not substitution (measured by job loss).

Opinion: And yet Anthropic are still predicting 70% of jobs exposed and are not naive, and yet junior devs are –15%. Overall I just think we’re early and do expect true mass exposure within 10 years.

Cursor releases Composer 2.5, nominally matching Opus and GPT on SWE-Bench and CursorBench at 1/8 cost. Built on Kimi K2.5, but 85% of the model’s total compute was spent entirely on code-specific training and RL. In future: SpaceX will enable a significantly larger successor model, 10x more compute.

Opinion: We must not pay attention to SWE-Bench anymore.

Capabilities#

Some of our predictions for Gemini 3.2 (out later today?):

  • amazing knowledge and math benchmarks, amazing OOD vision and audio

  • but, behind the nominal performance, still not frontier-frontier.

  • Two weeks of buzz followed by usual widespread nonusage

  • Maybe (20%) some splashy implementation of “nested” (i.e. expensive continual) learning.

Google’s Aletheia, a math AI powered by Gemini, autonomously solved an open problem from the Kirby list (renowned compendium of important problems in topology). Some context making the finding less explosive: out of 46 problems tested, the AI successfully answered one by pulling from decidability theory, which the researchers called a “lucky find of low-hanging fruit.” For the other 45, it failed 38 times, exploited flawed problem statements 5 times, and merely cited a recent paper for another.

Opinion: Clearly Aletheia didn’t solve ‘one of the most important open problems in topology’! Those famous-mathematician-name lists are extremely heterogenous. Still, achieved something outside of the usual Erdős-flavored area. Friend from GDM mentioned yesterday that in January GDM brought in an algebraic geometrist/topologists to teach Gemini algebraic topology and topology, so makes sense we’ll see some fruit around now.

Context rot is still very real for classifiers: new Anthropic research finds that the models used to monitor agent transcripts for dangerous actions in e.g. Claude code struggle when malicious actions are inserted inside/before a long benign transcript.

Opinion: Classifiers have to be cheap (and, ideally, fast). Context rot is known to still be an issue at context lengths larger than 200–400k tokens even for frontier models. Smaller models – like the ones used as classifiers – are going to be hit harder.

Forecasting benchmarks should be awesome: fully decontaminated by definition, real-world valuable target, constantly refreshed. New benchmark seems like a move towards making them core evidence: <25% accuracy and nearly random Brier. GPT and Claude way ahead, as they should be. They find a dramatic inference scaling effect, with no plateau??

Opinion: I mostly buy it. Many ways to do this wrong, but they’re trying. They seem to be taking knowledge cutoffs naively, as hard thresholds, but claim that performance didn’t drop over time.

AI evaluation is structurally unsuitable for continual learning. Paper proposes a new kind of eval, which includes:

  1. “landscape characterisation”: what behaviours can the system reach & with what probability?

  2. “trajectory forecasting”: how will a deployed instance evolve from its current state?

Paper has a nice taxonomy of continual learning: CL1 (within-session ICL), CL2 (external storage across sessions), CL3 (weight updates across sessions).

Opinion: A major headache, and one reason I expect to get more uncertain about system capabilities in ~a year. But this is a good idea.

Excellent nontechnical thread explaining our confusion at what AI can do. Attempt to reconcile the “AI is incredibly powerful already” and the “AI is overhyped and doesn’t do that much” positions by pointing out that both of these are happening, but by different people and in different circumstances.

Opinion: Endorsed, send this around to your power-user friends and sceptic friends alike!

Great causal experiment on LLM myopia: while models do write out lookahead when evaluating moves for a simple game, they don’t act on it; despite performance increasing with search effort, search depth 1 was sufficient to recover their final decisions. “LLMs’ search is shallower than humans’, and that performance is predicted by search breadth rather than depth”

Opinion: Myopia is one way to have pretty safe AI! Amusing if all this CoT optimization just makes them really good at talking like they’re not myopic. However, frontier LLMs are master level at chess now, which implies that they must be looking quite a bit ahead (4–5 nominal plies).

Recap of recent changes to the architecture of SOTA open models. Mostly: Highly targeted changes to the transformer block to reduce the cost of long-context inference.

Opinion: Long-context hacks have imo somewhat degraded performance on certain things like deep reading of long input like novels (while obviously helping a lot with large codebases with lots of local structure). This has been clear in closed models over the last two years (which saw an increase in nominal context windows from 64K → 1M). But one could imagine that the frontiers have clever ways to route exact attention for short inputs and approximate attention for long ones.

Tom Reed discusses the limits of “time-horizon” accounts of AI capabilities, highlighting that variation across domains and specifically between tasks with test sets evaluating partial progress along the way make for a lot of jaggedness and reduce the usefulness of the metric.

Opinion: Already internalized by P3, but this (short) discussion has a nice account of both why the METR graph used to mean something and why it no longer means much.

New blog post finds phase transitions in tasks relevant to in-context learning during pre-training: for doing simple arithmetic in spite of distractors, “OLMo3 32B hits 81% accuracy at 2.17T tokens, collapses to 0% at 2.19T tokens, then rebounds to 81.7% at 2.21T tokens“. These probes enable the authors to predict a given checkpoint’s generalization: checkpoints that display the most robust in-context learning also perform the best on out-of-distribution benchmarks later.

Opinion: Somewhat unsatisfying and special-case. For some apparently ICL-relevant probes, pre-training is non-monotonic for some model/probe combinations, and that based on a single benchmark and a single model, these probes can be leveraged to obtain better OOD performance. Your favourite lab’s mileage may vary!

Politics#

Annals of mass discontent: US populace is the most worried in the world about AI (50% net-negative, 70% against the current pace). Commencement speakers (including Eric Schmidt) getting booed for cheerleading it.

Opinion: This is what it’s like when only really junior SWE is actually hit. Mildly impressed that the populace can extrapolate (though their prior is against big changes in general).

Anthropic-DoW oral arguments finally begin. Henderson: “I see no evidence of sabotage, malicious, etc. I don’t see any evidence that the DoW has supported its designation that Anthropic is a bad actor. To me this is just a spectacular overreach by the department”

Gov justification: “Needed to act quickly; Anthropic’s model was deployed on multiple DoW platforms; SCR was the way to get the entire department aware of the risk [posed to operations by Anthropic].”

Opinion: Too early to say.

Non-news: Basically nothing public came from the Bessent-China AI talks. Beijing has “agreed to launch an intergovernmental dialogue on artificial intelligence”.

Opinion: Verbal restart is still something, ok. Heavily inferring, the focus seems to be on keeping 10Ts out of nonstate actors’ hands. It’s a start I guess.

Rare ideological intervention from a Chinese lab (Moonshot). Calls Anthropic irresponsible, defends open-source as decentralisation of power.

Opinion: It’s fair to be cynical about how much a Chinese public figure can really believe in decentralisation of power. But he’s free to argue the narrow sense, of reducing US relative power. And he must know that the “Anthropic” policy is being pushed by USG.

The Netherlands objects to possible US law which refers to Dutch (i.e. ASML) exports to China; no actionable consequences yet. China was a third of ASML’s sales in 2025. Their EUV has never been shipped to China.

Opinion: Obvious defence of sovereignty / correct bargaining move, but they complied with a narrow 2022 control on their US staff. And note that it’s not really “China”; the bill names CXMT, Hua Hong, Huawei, SMIC, and YMTC.

Safety#

Good thread about one of our cruxes: AI is mostly improving at verifiable things, but alignment is not verifiable, so we should expect to be in trouble for simple differential progress reasons.

Opinion: Classic argument, though the (OAI) poster is asking for evidence for it, he’s not yet convinced.

Odd Dwarkesh essay about intelligence not being tightly linked to power in humans. “in the real world, power is more the product of having the authority and trust to get lots of people to collaborate with you, rather than some galaxy brain scheming capability”.

Opinion: OK, but are the intelligent better at getting that authority and buy-in? Yes.

Paper on steering vectors finds a reused preference vector shared by model personas, even in personas opposite to the vector’s (e.g. an evil persona vs the “helpful assistant” persona).

Opinion: Marginal progress in pragmatic interpretability and linear representations; not especially surprising or novel.

Microsoft red teaming vulnerabilities in agent networks; overall, inadequate verification and too-high default trust. A small number of agents naturally developed privacy-protective norms.

Opinion: The headline appears to be misleading, since several of the issues need an individual agent to not be safe and the network merely amplifies/fails to fix the damage. Otherwise nothing surprising - the default expectation is that agents are exploitable by sophisticated attackers.

You can push activation distributions through the network to identify misbehavior in expected outputs, including tail risks, without running samples through the network. This could allow developers to squash dangerous behaviors missed by standard empirical training data and helping against deceptive alignment.

Opinion: Very good step; authors admit that lots of work still has to happen, but this is part of a promising agenda with big potential payoff. The “without running samples” part is the important one.

Argument that fine-tuning which treats a model’s past outputs as evidence about the world is systematically doomed; shows that classic SFT reliably produces delusions, but that their interventional SFT avoids them.

Opinion: Sensationalized and the authors generally oversell how easy this genre of intervention is, but this is still impressive and the theoretical basis of distinguishing a model’s past outputs from other data is both sound and useful. There’s also a tenuous extension towards identifying how many layers of synthetic data generation and training-on has occurred overall, which is currently out of reach but might be vital if model collapse turns out to be a major concern.

Incidents#

Musk’s lawsuit against OpenAI and Microsoft for the charity-into-for-profit switch fails due to statute of limitations. The defense successfully argued that Musk had access to all relevant information in 2017–2019 and the 4-year statute for breach of charitable trust had expired by 2024, when the lawsuit was filed. The plaintiff argued that the full bait and switch did not become apparent by 2022, but the advisory jury was not convinced and the judge adopted their verdict. Appeal coming.

Opinion: Broad outcome expected — why we’ve kept this stuff in ‘minor’. 15% on appeal succeeding in some sense.

Minor#

  • Karpathy joins Anthropic. I give him a year.
  • Anthropic’s natural language autoencoders fail to interpret steered activation states faithfully. Also shows that steering pushes the model into states that are mathematically impossible to reach using standard text inputs.
  • How do Chinese labs distill? Clever!
  • ARM produces their first datacenter CPU, gets an antitrust investigation in response.
  • Orthrus: inject a trainable diffusion attention module into each layer of a frozen AR Transformer sharing a KV cache.
  • Attempt to fix LLM writing style by retargeting towards making outputs more similar to training data as opposed to some vague notion of “better”; at the very least finds that the tool fools AI text detectors.
  • Argument against last week’s meme of deleting LessWrong from training data as alignment intervention. Would leave models less capable of understanding moral reasoning, and the content is available in other fictional media (ie Grima Wormtongue); therefore, don’t scrub, and instead introduce positive examples into training data.
  • Zyphra release alleging 4–7x speedups with minimal performance loss. Skepticism warranted.
  • Nvidia’s SANA-WM: further advance in video generation.
  • Tour of Jane Street’s datacenter. Not leased from hyperscalers?? How many other obscure ones are there?
  • Blog post by alignment research skeptic arguing that the traditional fears (instrumental convergence, fast takeoff) are at odds with AI architecture and our observations of biology. Doesn’t appear to understand the alignment-minded position, but seems to engage in good faith.
  • Obviously AI short story wins a Granta Commonwealth Prize
  • Discussion of the stifling effects of employment at frontier labs on ability to talk about risks from AI freely. Something of a counter.
  • Singapore’s Foreign Minister talks about how he built his diplomacy exo-brain via Claude & Whatsapp on a Raspberry Pi.
  • Post on the failure of AI to do novel science or tell good jokes relying on Ord’s concept of hyperpolation. Key question, related to our vertical generalisation crux.
  • Observation of AI recursive self-improvement disagreements; highlights cruxes such as likelihood of future AIs becoming more human-like and observed correlations such as prior work experience only in AI research or small teams occurring alongside high expectations from synthetic data.
  • Controversial Richard Ngo thread on which kinds of safety work have turned out to be relevant and the value of theories of impact. “EA impact reduces to Anthropic impact”.
  • AI R&D anecdote: Codex weakened a test; an autonomous Claude noticed this, and decided to un-weaken the test.
  • Anthropic post on using Claude Code in large codebases.
  • New continual learning algorithm which works during all parts of training; matches SOTA at reduced cost primarily via LoRA interpolation.