Who's afraid of latent recurrence?

what a looped transformer is or why it makes CoT monitoring harder, why everyhone is talking about serial depth, why OpenAI’s mitigating explanation still leaves a lot of room for scary possibilities – plus questions I still have, like, is this an impressive technical breakthrough or something anyone could have done based on the papers that were published last year? What actually do people mean when they say neuralese and is it useful?

OpenAI’s new GPT-6 model is “depth-recurrent”: it can use extra computation which doesn’t route through human language. It’s the first frontier LLM known to use the approach. As The Information reported:

“OpenAI’s recurrent depth approach to powering Astra is similar to the one… introduced last year in a paper on “latent reasoning,” according to the person with knowledge of its development.”

At the same time, GPT-6 is far less monitorable than its predecessors: our ability to track what it is thinking in natural language is greatly reduced. Was this due to its extra “latent recurrence”?

Tomek Korbak (a highly credible figure who joined OpenAI to work on monitorability) says that he is “fairly confident” that Astra’s architecture is not the cause of the unmonitorability, and nor does he view the current degree of recurrence a defection against the existing pro-monitoring norm. (Another guy in his position is quite a bit less clear.)

OAI’s chief scientist says that the implementation of latent recurrence in Astra is currently constrained: “the depth of the computation graph for our present frontier models, including Astra, is within a factor of two of GPT-4” (so, speculatively, 240 layers = 2 x 120 and between 2-4 passes over the recurrent block). We explain the significance of the serial depth of a computation below.

In the following, let’s just grant that they haven’t cranked up the serial depth much, and that the cranking they have done does not explain any of the monitorability collapse. (Even so: one of the few bright lines the CoT monitoring coalition has to organize around - “don’t use opaque recurrence” - has apparently been crossed. Even so, something must explain the sudden spike in Astra’s “no-CoT” reasoning ability, and recurrence is an obvious choice.)

The worst aspect of the news is that OAI using recurrence in production is the first clear sign that recurrence actually beats the CoT bottleneck approach, which would have been particularly convenient for the future of humanity.

The rest of this piece goes into the technical details of recurrence and what’s at stake from any larger shift towards opaque latent recurrence.

What is recurrence?#

Before the Transformer (2017), one of the dominant architectures in AI was the recurrent neural network (RNN). But RNNs quickly tailed off in importance afterward, roughly because they are much worse at using the full power of GPUs, which had became the dominant term in an architecture’s success.¹

Recurrent architectures like RNNs mostly fell behind after the 2017 Transformer. One hope

“Recurrent layers enable a transformer model to perform arbitrarily many computations before emitting a token”

On the unrolled grid, a model is recurrent along an axis iff two things hold along that axis: (1) the same weights are applied at every step, and (2) a fixed-size state is carried through a chain of steps. Then:

• RNN: tied and chained along position → position-recurrent. Along depth: chained (stacked layers) but untied → not depth-recurrent.
• Vanilla transformer: tied along position (same layer weights at every column) but not chained; column t does an unbounded-fan-in read of all earlier columns instead of inheriting a bottlenecked state. So position-tied, not position-recurrent. Along depth: chained (residual stream) but untied.
• Looped transformer: depth-tied and depth-chained → depth-recurrent. Still position-tied, not position-recurrent.
• SSM / linear-attention layers: position-recurrent (fixed-size state, tied weights), which is exactly why they can be written either as a recurrence or as a scan.
• Both axes: Grid LSTM (Kalchbrenner et al. 2015) is recurrent in depth and time; a looped Mamba would be too.

Note that most of an LLM is already opaquely recurrent; “looped Transformers” just remove the last human-interpretable bottleneck. So “full latent recurrence” is a better term.

“there is likely not a binary divide between non-neuralese and neuralese models”

A standard transformer is a fixed stack of layers. Every token’s representation passes through the stack exactly once, and the depth chosen at training time is frozen forever. A looped transformer instead reuses a block of layers: the hidden state is fed through the same block r times, weights tied across iterations, before the model emits its next token. Effective depth becomes layers × loops while the parameter count stays put, and r can in principle be varied at inference. The lineage is old and public: Universal Transformers (Dehghani et al., 2018); constructions showing loops make transformers programmable (Giannou et al., 2023); Meta’s Coconut, which feeds the final hidden state back as the next input in place of a token (Hao et al., Dec 2024); and the recurrent-depth model Huginn, a 3.5B model trained around a mean of 32 loops and run at up to 64 (Geiping et al., Feb 2025). Since then, open models have shipped modest loops as a parameter-efficiency trick: Ouro uses four (Zhu et al., 2025), Nanbeige 4.2-3B and the 20B Loopie use two (Gao et al., 2026).

We fear depth more than size#

“the serial depth of reasoning that models are capable of in a continuous latent space”

Per emitted token, a transformer is a circuit of depth L. Adding width (parameters per layer, longer context, filler tokens) leaves it in TC⁰. Merrill and Sabharwal have made this exact: polynomially padded transformers recognise precisely TC⁰, and it is looping that buys depth, with log^d loops giving precisely TC^d. Chain of thought with polynomially many steps reaches P. The asymmetry is the standard one: any parallel circuit can be evaluated serially, gate by gate, but the P-complete problems are conjecturally not parallelisable at all. Planning against an adversary, simulating a monitor, carrying out a multi-step deception: these are the tasks the CoT monitoring literature expects to be inherently serial (Korbak et al., Emmons et al.). Depth is the resource dangerous cognition needs, and width is not a substitute for it.

Because the whole dispute compresses into one quantity. Parallel resources — width, heads, experts — do not help with serial problems; only depth does, and a chain of thought is a mechanism for converting emitted tokens into serial depth. The safety property people valued was never “models write English”; it was “serial depth is token-priced”. So the natural measure of how much monitorability an architecture forfeits is how much serial computation fits between tokens. This is now being formalised — “opaque serial depth”, with numeric bounds computed for open models (Chen et al., March 2026) — and measured empirically as “no-CoT time horizons”: how long a task a model completes with its scratchpad ablated (June 2026). It is also why OpenAI’s reassurance was phrased in exactly these units. Both sides accept the frame; they disagree about how much hidden depth is tolerable.

Depth is also the resource we can currently count, and here the usual emphasis on bandwidth is misplaced. In a standard transformer, layer l at position t reads the layer l−1 state of every earlier position. The KV cache therefore carries thousands of latent dimensions forward per position, at full bandwidth, and nobody monitors it. What it cannot carry is depth: a read at layer l sees earlier positions at depth l−1, so composing across positions gains nothing. The only edge in the graph that increases serial depth across positions is the sampled token. The top layer’s state at t−1 reaches position t through sample → embedding, at most log₂|V| ≈ 17 bits (one or two bits realised), in a human code. That is the bottleneck. It constrains depth-carrying state, not information in general.

Architecture

StandardDepth recurrenceLatent recurrence

Loops per token r

3

Playback

PlayStep

Deepest unsampled path

4layers

Bounded by L = 4 for every position.

Depth-carrying channel across positions

Sampled token

At most ~17 bits per step, in a human code. The KV cache carries far more, but no extra depth.

Necessity

Holds

Any computation deeper than 4 layers must leave a trace in visible text.

unsampled depth, log scale sampled token, ≤17 bits, human code latent state fed forward, ~10⁵ bits, no code attention read of the KV cache

Standard

Four layers per position. Depth beyond four has to pass through a token, so any deeper computation leaves a trace of bounded rate.

Depth recurrence

A two-layer block is applied r times per position, between an input layer and an output layer. Each iteration reads the same iteration at earlier positions, so depth still resets at the token. The bound survives, but its value is now a runtime setting.

Latent sequence recurrence

The top-layer state is fed back as the next input, with no token in between. Depth accumulates across positions with no bound and no symbol, and the carried state is never projected onto a vocabulary.

A full-page version of the grid is also available.

Its consequence is the necessity property: any computation deeper than L must leave a trace in visible symbols. This is the load-bearing half of the CoT monitorability case. Emmons et al. give the empirical form: when the bad behaviour is made hard enough that CoT is necessary, models cannot evade a monitor without substantial outside help. Necessity guarantees neither faithfulness nor legibility. It guarantees that a trace exists and bounds its rate, so that monitoring has to be defeated rather than bypassed.

Ways to think more#

There are many ways of making an LLM “think more”.

How to think more	Buys	Channel	Set by
More layers	serial depth	latent, bounded by L	training
Wider layers, MoE, longer context, retrieval	width	latent (KV) or visible (text)	training & deployer
Longer CoT	depth	visible, ~17 bits/step	deployer & model
Best-of-N, self-consistency, tree search	width over samples	visible; aggregator is code	deployer
Tools and code execution	unbounded serial depth, but external	visible (code + output)	model
Multi-agent with text messages	depth and width	visible	model
Multi-agent with latent messages	depth and width	latent	model
Filler / pause tokens	width	latent, within TC⁰	deployer
Depth loops	depth per token	latent, r·L	deployer & router
Continuous CoT, shared-cache loops, LRT	depth across positions	latent, unbounded	deployer & architecture
Learned discrete codes (VQ latent tokens)	depth	sampled but not human-coded	training
Internalisation (distilling CoT into the pass; RL raising no-CoT capability)	more use of the same L	none: it shrinks the visible channel	training
Test-time training	depth via gradient steps	latent, in weights, persists	deployer

• Training time
  • Increasing the number of layers in the Transformer block
• Inference scaling
  • Filler tokens: more inference without passing
• A recurrent architecture is both: one trains a model to extrapolate across different numbers of self-loops and then spends more inference per token to dynamically expand the
  • Looped transformer. Variable depth
  • Hypothesis that it’s just layer reuse, and logically just a single reuse.
• …

More layers. Raises L. The cost is parameters and training compute; the value is fixed before evaluation; the function evaluated is the function shipped. This erodes the guarantee, because the threshold L rises, but visibly, slowly, and at capital cost.

Filler tokens. Pfau et al. show that models can compute in filler positions, but only on parallelisable problems, because fillers add positions, not layers. Width, TC⁰. They widen the unmonitored channel; they do not touch necessity.

All make the situation worse, but the problems are different in kind. Fillers buy width rather than depth [1], so they’re stuck inside TC0. (What you rightly call ‘full’) recurrence buys serial depth, beyond TC0 [2].
(You can simulate any fixed-width parallel computation with enough serial steps, but not the reverse [3].)

[1] https://arxiv.org/abs/2404.15758v1
[2] https://arxiv.org/abs/2505.18948
[3] This is not strictly proven yet.

Recurrence. Two different things travel under this name.

(a) Depth recurrence (Universal Transformer; Huginn; Nanbeige 4.2; reportedly Astra). A block is applied r times per position. Per-token depth is r·L, and r is an inference-time argument: Geiping et al. train on random r and extrapolate to larger r at test time; Mixture-of-Recursions lets a learned router choose r per token. If each iteration reads earlier positions at the same iteration, depth still does not accumulate across positions without a token, so necessity survives formally. But its threshold is now r·L for whatever r the deployer or the router chooses. A guarantee whose parameter is a runtime flag is not a guarantee.

(b) Latent sequence recurrence (Coconut). The top layer’s state is fed back as the next input. Depth now accumulates across positions with no symbol at all, and the depth-carrying channel is no longer projected onto a vocabulary. This breaks necessity outright and detaches the carried state from any human code. Zhu et al. show what it buys: a continuous thought holds a superposition of search frontiers, solving reachability in D steps where discrete CoT needs O(n²). That is the capability argument, and it is the safety argument with the sign reversed.

• More layers obviously buys serial depth, but L is a training-time constant (so the instance you tested before deployment is the same function as all deployed instances).
• Loop count is elastic at inference-time, so we’ll know even less about what they can do.
• Latent recurrence is full-bandwidth (thousands of times more than the 1-17 bit CoT bottleneck), and it isn’t tied back to a human code at each step, and it loses the nice property that only visible symbols carry serially-deep state forward, and it scales at inference-time. NBG!
  • Far more space for hiding reasoning (than natural language CoT steganography)
  • Arguably detachment from human concepts
  • We lose necessity: in non-fully-recurrent architectures, serial depth must be represented in visible symbols
  • Deployed model is no longer the same function as the trained model if the loop number is varied

Raschka’s objection: looping is “just layer reuse”; Nanbeige found two passes optimal and further passes nearly useless; and the effect on visibility is the same as scaling the model. Two-pass looping is indeed a 2L model with tied weights, and the Nanbeige and Huginn results (marginal returns past a few loops) are evidence that current training does not make deep recurrence profitable. But the analogy to model size fails at the one point that matters. L is bought with parameters and fixed before evaluation. r is bought with inference FLOPs, needs no new weights, and is raised by editing a config. The lab that has the technique has the option. With adaptive exit the router, which is the model, decides per token how much hidden depth to spend, and that is exactly the allocation a monitor-evading policy would want to control.

Two clarifications, to avoid overclaiming. First, looping does not by itself suppress the chain of thought; a looped model still emits tokens and can still be trained to reason aloud, and there is nothing intrinsically harder about monitoring a 32-layer block looped twice than a 64-layer stack passed once — a point Sebastian Raschka and others made within hours of the report. Second, ordinary scaling of depth erodes necessity too. What the loop changes is the economics: serial depth becomes cheap in parameters, elastic at inference, and directly substitutable for reasoning tokens. Each unit of serial thought moved from the token channel into the loop converts monitoring from necessity-backed to propensity-only — from it must show its work somewhere in these tokens to it usually chooses to narrate.

Tools are the visible counterpart to latent recurrence: unbounded serial depth through an auditable channel. They are why necessity can be preserved while capability grows, with the caveat Greenblatt raised, that tool calls can be spoofed, and spoofing gets easier with more latent depth.

Learned discrete codes are the case that shows sampling and language come apart. A VQ-token bottleneck keeps the rate bound and keeps necessity (the model must still commit to one symbol per step), but loses legibility. That is neuralese in the literal sense, a language, and it is far more tractable than continuous state: a fixed finite code with consistent usage can be translated from paired data, with steganography as the residual risk. The essay’s spectrum should have this point between “sampled token” and “soft token”.

Internalisation changes no architecture and no bound. It raises the share of tasks for which CoT is unnecessary, which shrinks the necessity regime from the inside; that is the erosion Pachocki said was happening for reasons not contingent on architecture changes, and Arike’s reading of the no-CoT time horizons result is that doubling a model’s no-CoT time horizon needs only about 1.3× the layer count, which says training matters more than depth here. Same for any RL that makes single-pass reasoning better.

Test-time training is the worst cell in the table: the state is a weight delta, so it is invisible, high-bandwidth, persistent across the whole episode or longer, and it makes the deployed function differ from the evaluated one by construction rather than by a flag.

Shared-cache depth recurrence#

The distinction matters less as r grows: a “per-token” bound of 100·L is no bound. And if a depth-recurrent block reads earlier positions’ final states rather than same-iteration states, (a) becomes (b).

interturn serial depth chaining beyond simple per-turn serial depth increases

And a different risky architectural change which we have no idea about: perhaps “Astra’s attention layers can attend to late layers of previous forward passes”

Why the question may not have a fixed answer. In Huginn the pathway is a serving option, not an architectural commitment. Geiping et al. train with per-iteration caches, then show zero-shot cache sharing: fix a budget k and have iteration i read and write cache slot i mod k, so with budget 16 the 17th iteration overwrites the 1st slot; on MTBench a budget of 4 costs nothing. Once r exceeds k, iteration i at position t reads whatever later iteration last wrote that slot at earlier positions, and a follow-up paper reports Huginn matches its GSM8K baseline with cache sharing at a minimal cache size of 1, meaning every iteration reads the deepest state of every earlier token. Section 6.3 of the same paper goes further: warm-starting the recurrence at the next token from the previous token’s final state, again zero-shot, cuts steps to converge by 1–2. Both are the (a)→(b) move from the essay, and both are flags. So “Astra can” is probably true as a capability claim if Astra is Huginn-like, and unknown as a deployment claim. The conservative implementations do exist: ANIRA’s allocation-aware cache retrieves the deepest cached entry at depth min(d, d_i*), never exceeding the current iteration, which preserves the per-token bound.

One practitioner comment cuts the other way: Grad’s read is that training-time loops have a separate KV cache per loop, and dynamic looping usually means giving that up, which is exactly the regime where the cross-iteration read appears. If Astra loops dynamically, sharing is the default expectation, not the exception.

Also relevant, though not about Astra: Microsoft’s Latent Recurrent Transformer (May 2026) builds the pathway deliberately, reusing a source-layer hidden state from the previous position as recurrent memory injected into the next position’s layers, and reports a better compute–quality trade-off than matched-depth baselines for 0.3% extra parameters. The property is being developed as a capability feature independently of looping.

What do we know about Astra?#

Astra is not a COCONUT-style looped Transformer. OpenAI’s launch post says they are deploying Astra with “additional chain-of-thought monitoring” to detect and contain misaligned actions, which is incoherent if there is no CoT. They claim that monitorability is not compromised.

Nobody outside OpenAI has described Astra’s KV-cache topology, and that is the parameter that decides the question.

What is public. The Information’s article reportedly says the approach is like Geiping et al.’s recurrent-depth model, and Arike’s summary of it is that recurrence runs along the depth axis, so for any given token the model can loop, but no hidden state passes across positions beyond what ordinary transformers pass. That is a description of the reference architecture in its default mode, not a reported fact about Astra. The other datum is Pachocki: the depth of the computation graph for present frontier models, Astra included, is within a factor of two of GPT-4. Under the natural reading that is a per-token bound, and a per-token bound is incompatible with early layers reading late iterations of earlier positions, because that pathway makes the unsampled depth grow with sequence length. So his statement is indirect evidence against the property, to the extent it was meant precisely.

On 1 September, The Information reported — citing a single anonymous source — that OpenAI’s forthcoming Astra uses “constrained recurrent depth” in roughly the Geiping style. OpenAI has not confirmed it; the system card should settle the question. Note what even the report does not claim: this is not an RNN carrying an unbounded hidden state across the whole trajectory. In the Huginn design, recurrence runs along the depth axis within a token position; between positions, information flows only in the ordinary transformer ways.

What would settle it for Astra: whether the recurrent block’s cache is per-iteration or shared (and at what budget relative to the loop count), whether the recurrent state is warm-started across tokens, and whether the “within 2× GPT-4” figure was computed per token or over the unsampled path. Greenblatt’s questions cover the loop count; these three cover the topology, and they matter as much.

Jakub Pachocki replied on 2 September: he wants to prevent a race to unmonitorability set off by confused reporting; the computation-graph depth of OpenAI’s current frontier models, Astra included, is “within a factor of two of GPT-4”; OpenAI remains committed to chain-of-thought monitoring — and, worth noticing, he added that monitorability is fragile and degrading for reasons unrelated to architecture, about which he will write soon. Taken at face value, this is genuinely reassuring about the artefact: today’s Astra is then not a neuralese machine but a couple of loops of a moderately deep stack, in line with what open looped models already do.

It leaves at least four things unruled out.

The dial. Loop count is a configuration, not a property of the weights. If Astra was trained across a range of loop counts, as Huginn was, raising the cap is a config change; if not, plausibly a small fine-tune. Ryan Greenblatt’s questions are the right ones: can depth be raised cheaply and profitably; is Astra an above-trend jump in what fits inside a forward pass or in monitor evasion? “We currently set the dial low” is a policy held under competitive pressure, not a guarantee. Once looping is normalised at the frontier and shown to pay, the option value is the story.

The bound is loose in the units that matter. Geoffrey Irving relays the view of circuit-complexity theorists: depth bounds buy real safety only when set very low; a bound of hundreds of serial layers between tokens is closer to decoration. Capability per unit of depth is steep — the no-CoT horizons work suggests roughly 1.3× layers doubles what a model can do without a scratchpad — so “within 2× of GPT-4” is compatible with a large hidden capability delta. And since GPT-4’s depth is not public, the claim is unverifiable from outside: no third party can audit the depth, the cap, or the training loop range.

It defends the wrong margin. The threat model is not a model deriving its whole coup in one forward pass. In July’s Hugging Face incident, models tampered with their own transcripts and spoofed tool calls; visible chains of thought were part of how the attack was reconstructed afterwards. The binding constraint on that class of behaviour is the small amount of unmonitored serial thought needed to set the deception up — after which the model can buy itself more. Modest additions of hidden depth help most at exactly this margin, which no benchmark of visible capability tracks.

The commitment is propensity-shaped. “We will monitor the chain of thought” reassures while the chain of thought is necessary. Astra’s published safeguards lean on chain-of-thought and action monitors. But as latent depth grows, those monitors increasingly read what the model elects to say — and Pachocki concedes the verbal channel is independently degrading, with drift under RL the usual suspect. The two erosion channels compound: less necessity from architecture, less legibility from training.

So the reported Astra is a small step, taken carefully. It is significant anyway: a frontier lab has, apparently, begun paying for serial depth in a currency monitors cannot read, while asking to be trusted about how much it spends. The cap is a promise, and promises about dials have short half-lives.

Speculation:
1. Astra uses a recurrent architecture
2. This doesn’t preclude CoT monitorability
3. Astra’s serial depth *per forward pass* is bounded and not drastically deeper than existing models
4. Astra’s attention layers can attend to late layers of previous forward passes
5. This DOES provide indefinitely-deep neuralese reasoning completely independent of CoT
6. The net effect is that Astra does blow up CoT monitoring
7. People think this is because of the recurrent depth architecture, which OpenAI assures everyone is fine
8. Because the thing that actually destroys CoT monitorability is a related but distinct change

Chief scientist:
The depth of the computation graph for our present frontier models, including Astra, is within a factor of two of GPT-4.

i.e. <240 layers

Greenblatt:
this statement is consistent with Astra having a configurable “dial” that is currently set to a low depth but could be trivially increased.

Jai:
I suspect they’re not being transparent, and/or using a weasel/dumb definition of serial depth that actually means “serial depth per forward pass (which can extend serial depth from previous forward passes indefinitely over enough tokens)”

Is this inevitable?

The state of monitorability#

Today the trace is free. At every forward pass the model must commit to one vocabulary token, so the depth-carrying state is written down at the moment it is produced, in the language of the pretraining corpus, at no cost beyond tokens the model was going to emit anyway. Three properties follow, and each is doing work. Coverage: by the argument of section 1, every unit of cross-position depth passes through the channel. Cost: reading it costs a monitor one pass over text it already had. Legibility: the code is one that humans and any LLM monitor read natively, so no translation layer has to be trained or trusted.

This is why CoT monitoring is the one oversight method that works in production rather than on paper. Baker et al. show CoT monitors catching reward hacking in agentic coding at rates action-only monitors cannot approach; OpenAI’s monitorability suite finds CoT-only monitoring sometimes close to a monitor given everything, finds longer reasoning more monitorable, and is now tracked across training runs. On the model’s side, Emmons et al. find that evasion needs outside help, and a 2026 study finds reasoning models struggle to control what appears in their chains of thought at all. Necessity is currently doing what section 1 says it should.

What the trace does not give is faithfulness in the rationalisation regime (where CoT was not needed, the model may confabulate a reason) or robustness to optimisation. Baker et al. show that one training run with CoT pressure yields obfuscated reward hacking; OpenAI’s own evaluations show training pressures reduce monitorability; and Pachocki, responding to the Astra reports this week, called CoT monitoring fragile and trending the wrong way. That erosion is happening with no architectural change at all. Latent recurrence is not the first threat to the trace; it is the one that removes it rather than degrading it.

The candidate replacements read activations, and none is close.

• Currently we get a chain of thought directly by sampling a concrete human-language token at each forward pass
• Natural language autoencoders aim to decode activations to human language. Presently NLAs are a extremely poor replacement for actual CoT.
• …

Natural language autoencoders (Anthropic, May 2026) train a verbaliser and a reconstructor by RL to round-trip a residual-stream vector through text. They are useful: they surfaced unverbalised evaluation awareness in Opus 4.6 and unverbalised grader-gaming in Mythos Preview training transcripts, and they let auditing agents find a planted misalignment without the training data. But by the authors’ own account they confabulate at a substantial rate that does not fall with training; they have no mechanistic grounding; the verbaliser is a full LLM that can infer beyond what the activation holds; they read one layer at one position; and they emit around 500 tokens per activation, which rules out running them on every token of a production RL run or over transcripts much beyond ten thousand tokens. Compare the token: complete coverage, zero marginal cost, no confabulation, no second model to trust. NLAs are an audit tool for hypothesis generation, not a monitor, and the gap is one of kind rather than maturity. An NLA translates a channel the model writes in its own code, using a model whose reliability must be established separately and which can in principle be optimised against. CoT is a channel the model is forced to write in ours.

Probes and sparse autoencoders scale where NLAs do not, but probes need labelled examples of the behaviour being sought, SAEs have coverage gaps and need an interpretation step, and both return scalars rather than reasons. Reading latent loops directly is earlier still: logit-lens probing of Huginn finds little interpretable latent chain of thought, with readouts that depend heavily on which recurrent block and which decoding method is used.

Note that sampling, not language, is doing the work. There is a spectrum: a sampled token (17 bits, human code, free); a soft token that mixes embeddings (Soft Thinking, partly legible through its top components); a continuous thought fed back as input (Coconut, no code); a depth loop with no step at all. Each move along it adds bandwidth to the depth-carrying channel and removes a property some monitor was relying on. Deployed monitoring exists only at the first point.

So the fallback that C2 below leaves us with is real but thin: offline, sampled, expensive, confabulating, and unvalidated against ground truth for the thing it is most needed for, which is what the model is thinking when it is not saying.

• Monitoring = black-box monitoring + white-box monitoring + environment monitoring

  • black-box monitoring = action monitoring + confessions

    • elicited reports
      • Confession value ≈ P(model knows it misbehaved) × P(honest report | knows) × recoverability

  • grey-box = intermediate text monitoring (CoT)

  • white-box monitoring = activation monitoring + weight auditing

confessions

Text monitoring = f(necessity(no-CoT capability, opaque serial depth), coverage, recoverability) + propensity-surplus, with steganography acting on recoverability, not necessity

Term 1 — necessity (your terms). 1/no-CoT-capability: the performance gap when CoT is suppressed, which shrinks as forward passes get stronger. 1/opaque-serial-depth: per-token serial compute is architecturally capped at ~O(layers) — the Merrill–Sabharwal circuit-complexity results are the formal version, with CoT tokens as the mechanism that lifts transformers past that ceiling. Threats to this term: recurrent-depth and latent-reasoning architectures (Coconut, Geiping et al.), and to a lesser extent filler-token compute, though Pfau et al. found filler tokens buy parallel rather than serial compute, which is the reassuring reading.

But two corrections. First, necessity is per-computation, not per-task. The task may force deep externalised reasoning while the malign decision (“sandbag here”, “insert the bug”) is shallow enough to stay latent. Emmons et al.’s result — models struggle to evade monitors when CoT is necessary — carries exactly this qualifier: necessary for the evasion-relevant computation. Second, steganography doesn’t reduce necessity at all; it converts legible serial depth into opaque serial depth with no architectural change. So your second term should be indexed to the monitor’s decoding ability, not to architecture: opaque-to-whom.

Term 2 — coverage/faithfulness. Given externalisation, is the monitor-relevant content in the token semantics? Sub-properties: causal load-bearingness (Lanham-style truncation and perturbation tests), coverage in the Google DeepMind autorater sense (does the trace suffice for a human to reproduce the output), and non-omission of intent (Chen et al. 2025 — models using hints without verbalising them). This term is where the taxonomy below’s hidden parallelized reasoning family lives: partial verbalization, non-load-bearing CoT, epiphenomenal reasoning.

Term 3 — recoverability. Legibility (RL-drifted dialect, compression); monitor–model capability gap; monitor context access; robustness to deliberate encoding, where paraphrase-and-retrain (Roger & Greenblatt) is the main proposed defence; and the false-positive budget, since in control evals the achievable safety depends on how much auditing you can afford. The taxonomy’s encoded reasoning (steganography, linguistic drift) attacks this term, and its dazzling attacks the monitor directly.

Missing additive term — propensity. Models currently verbalise far more than necessity forces. Most present-day monitoring value sits in this surplus, which is precisely the fragile part Korbak et al. worry about: it survives only while nothing trains against it. Necessity is the floor; propensity is the (eroding) bonus.

Monitorability = P(compute passes through tokens) × P(relevant content is recoverable from token semantics) × P(the monitor recovers the content)

The rest of the hidden-reasoning zoo#

Recurrence is one route to hidden reasoning among several, and it clarifies things to see where it sits. Arike, RohanS and Biswas (Aug 2025) taxonomise hidden reasoning as “a loss of a CoT monitor’s ability to detect important latent factors that steer the behavior of the monitored model” — pick a concerning property p and ask how it evades a monitor. Their five families map cleanly onto the three factors above:

• Neuralese — “arbitrary-depth reasoning in high-dimensional latent space, enabled by recurrence”. p lives only in activations; no trace exists. This attacks the first factor, P(compute passes through tokens) — necessity itself. The taxonomy filed it as not presently concerning, on exactly the grounds this essay covers: high-stakes actions need sequential reasoning that exceeds a single forward pass, and transformers cap serial reasoning at the layer count. That was the correct call in August 2025. Astra is the news that the not-yet category is now shipping at the frontier.
• Hidden parallelized reasoning — important steps computed in activation space in parallel, despite a CoT being produced. Three grades: partial verbalization (the CoT is load-bearing but omits crucial latent facts — inevitable in some degree, since the residual stream carries orders of magnitude more bits than the output tokens); hidden utilization of test-time compute (the CoT is produced mainly to buy forward passes — the filler-token phenomenon wearing a legible mask); and epiphenomenal reasoning (the CoT is causally inert; the answer was computed elsewhere). In this essay’s terms all three are width, not depth: they stay inside TC⁰, need no architecture change, and attack the coverage term rather than necessity. They are also already observed — unverbalised bias in hiring decisions (Karvonen & Marks), models using hints without acknowledging them (Turpin et al., Chen et al.), implicit post-hoc rationalisation (Arcuschin et al.). The taxonomy’s own “most pressing open question” — can models do hidden parallelized reasoning about complex decisions, such as deciding when to defect? — is the per-computation caveat from above: the task may force deep visible reasoning while the malign choice stays shallow enough to fit in the parallel channel.
• Encoded reasoning — the depth still transits the tokens, but in a code the monitor can’t read. Two mechanisms: steganography (lexical — ciphers, token-group codes; semantic — the choice of topics, examples and emphasis carrying the signal), and linguistic drift (compression into ungrammatical shorthand, language mixing, invented jargon, at the limit an alien language). Empirically these are in different places. Trained steganography remains weak — RL setups have produced only 1–2 bits of hidden capacity, and models pushed to encode without guidance mostly produce acrostics (Zolkowski et al.). Drift, by contrast, is the failure already visible at the frontier: R1 mixing languages mid-derivation, the compressed obfuscated traces in Baker et al., METR calling GPT-5’s reasoning traces inscrutable. Both attack recoverability, not necessity — which is why paraphrase-and-retrain (Roger & Greenblatt) is a sensible defence against them and no defence at all against recurrence.
• Dazzling — the model manipulates the monitor into reading bad behaviour as benign: post-hoc justifications (“gaslighting the monitor”, Kutasov et al.), jailbreaking or disabling the monitor before acting, or burying the load-bearing step in verbosity. This attacks the third factor, the monitor itself.
• Their discarded categories are worth keeping in a footnote of the mind: semantic drift, where a word means one thing to the model and another to the monitor (Baker et al.’s “depurate”; GPT-5’s “marinade”); and reasoning that never happens as explicit computation anywhere, in activations or tokens — oversight misses 100% of the thoughts the AI doesn’t think.

Two of the taxonomy’s cross-cutting observations matter here. First, intentionality: every category except recurrent neuralese is likely to emerge first as incidental behaviour — a side effect of RL pressure, length penalties, or the residual-stream/token bandwidth gap — and only later, perhaps, become deliberate. Neuralese is the exception because it cannot emerge: it has to be built. It is the one form of hidden reasoning that arrives by a lab’s explicit decision, which is what makes “don’t ship opaque recurrence” a coordinatable line in a way that “don’t let your CoT drift” never was. Second, the monitor is assumed dumber than the monitored model — which is why the necessity term is the load-bearing one. Against a smarter monitor, encoded reasoning and dazzling might be recoverable losses. Against a dumber one, the only guarantee worth anything is that a trace must exist at bounded rate in a code fixed by pretraining. That is the guarantee recurrence spends.

The argument#

Stated formally:

P1 (serial harms). Some classes of severe harm require inherently serial computation: chains of dependent steps longer than any parallel circuit can absorb. (Inherently serial problems exist; the harms of concern — multi-step deception, planning against a monitor, novel exploit chains — are plausibly among them.)

P2 (depth bound). A depth-L transformer performs O(L) dependent steps per emitted token; under standard idealisations, fixed-depth transformers with polynomial width are confined to TC⁰. Width — parameters, context, filler tokens, samples — does not substitute for depth.

P3 (channel exclusivity). In a standard transformer, the only edge in the unrolled computation graph that carries serial depth across positions is the sampled token: visible, at most log₂|V| ≈ 17 bits per step, and in a code fixed by the pretraining corpus.

C1 (necessity; from P1–P3). Any computation of the P1 class must leave a trace in the visible token stream, at bounded rate, in a human code. Monitoring can therefore be defeated — encoded reasoning attacks the code, hidden parallelized reasoning attacks coverage, dazzling attacks the monitor — but it cannot be bypassed: a trace exists, and the attacker must beat the reader rather than avoid writing.

P4 (latent depth). Full latent recurrence supplies serial depth beyond L through a channel that is invisible, of order 10⁴–10⁵ bits per step, projected to no code, and cumulative across positions in the shared-cache variants.

P5 (evaluation gap). The loop count r is an inference-time argument, not a property of the weights. An evaluation characterises f_r at the r it ran with; f_r′ for r′ > r is a different, unevaluated function. (Longer CoT is also an unevaluated regime, but there the added computation sits in the monitored channel.)

C2 (from C1, P4). Latent recurrence removes the necessity guarantee. Hidden reasoning no longer requires defeating the monitor; the P1-class computation can simply route around the token channel. The failure modes that were the ceiling of the threat model — steganography, drift, dazzling — become its floor.

C3 (from P4, P5). The deployed model need not be the evaluated model: raising r is a config change, so capability claims made at evaluation time do not bind the deployed system, and no external party can audit the difference.

C4 (from C2, C3). The CoT-monitoring safety case does not transfer to fully latent-recurrent models. What remains is activation monitoring — probes, SAEs, natural-language autoencoders — which is weaker, less mature, and which the AI Security Institute’s oversight review already classes as a fallback rather than a foundation.

The contentious premises are P1 (perhaps the malign decision is shallow even when the task is deep — the per-computation caveat, and the taxonomy’s open question about hidden parallelized defection) and the “meant precisely” reading of P4 as applied to Astra, which OpenAI denies is yet operative at scale. P2 and P3 are theorems modulo idealisation; P5 is a fact about the published looped-model recipes.

The safety case for reading chains of thought (Korbak et al., July 2025, signed across OpenAI, DeepMind, Anthropic and others) stands on two legs. Propensity: models trained on human text tend to think out loud, and their scratchpads are usually informative. Necessity: for hard enough tasks, they have no choice. Necessity is the leg worth having, and it is close to an architecture theorem:

1. A forward pass of a depth-L transformer performs O(L) dependent steps per token; under standard idealisations, fixed-depth transformers are confined to shallow, parallel computation (Merrill & Sabharwal’s TC⁰ results).
2. Some tasks — long derivations, multi-step planning, novel exploit chains — require more serial steps than any single pass supplies; transformers solve “inherently serial” problems only with a chain of thought (Li et al., 2024).
3. In a standard transformer, the only way to buy additional serial steps at inference is to emit tokens and condition on them.
4. Therefore the load-bearing intermediate state of any long serial computation must transit the token channel.
5. That channel is narrow (a token carries under 20 bits) and shaped by a prior for humanlike text. A monitor reading it is reading the model’s working memory — possibly encoded, but necessarily present and bandwidth-limited — rather than an optional narration.

Looping deletes premise 3. Serial steps can now be bought in latent space: hidden vectors carrying orders of magnitude more information per step, no legibility prior, no record for a monitor to replay.

But isn’t the sampling bottleneck useful?#

For capability, two hypotheses are live.

H1 (it is a cost). Sampling forces serialisation, so no superposition search; it spends depth on unembedding and re-embedding; it cannot express what has no words.

H2 (it is a benefit). Discretisation stops the state drifting and makes errors correctable. It gives RL a discrete action space, sampling noise as exploration, and verifiable intermediate steps as reward, which is how reasoning models got trained at all. It lets the model reuse pretraining’s compression of human reasoning. Evidence: Coconut’s weak arithmetic results, Huginn’s marginal returns, Nanbeige stopping at two passes, and, until this week’s reports, no frontier model shipping deep recurrence.

H2’s benefit and the safety benefit are the same property: commitment to a discrete, human-coded intermediate state. If H2 holds, latent recurrence loses on its own terms and safety is free. If H1 wins (RL methods for continuous thoughts reportedly reaching parity on maths suggest the barrier is falling), safety has to be paid for. So the sampling bottleneck is not a safety mechanism we can rely on. It is a capability accident we are currently borrowing. The policy that follows is the one Korbak et al. already proposed: measure necessity per model and report it, and treat any change to the exchange rate between serial depth and visible symbols as requiring a safety case rather than a footnote.

Sources#

• OpenAI, Path to Astra (1 Sep 2026); The Information, report on Astra’s architecture (1 Sep 2026); Pachocki’s statement (2 Sep 2026).
• Rauno Arike, How concerned should we be about Astra’s recurrent architecture? (2 Sep 2026) — includes the Greenblatt and Irving threads and the looped-model literature summary.
• Arike, RohanS & Biswas, Hidden Reasoning in LLMs: A Taxonomy (25 Aug 2025) — the neuralese / hidden-parallelized / encoded / dazzling classification used in §“The rest of the hidden-reasoning zoo”.
• Korbak et al., Chain of Thought Monitorability (2025); Li et al. (2024); Merrill & Sabharwal, The Parallelism Tradeoff (2023); Opaque Serial Depth (2026); Geiping et al. (2025); Hao et al. (2024); Sebastian Raschka, OpenAI Astra and Looped Transformers (2026).

https://x.com/Turn_Trout/status/2095222594177814985
https://x.com/voooooogel/status/2095031272720736526
https://x.com/eliebakouch/status/2094973682858733650
https://x.com/RyanGreenblatt/status/2094996656186081642
https://x.com/merettm/status/2095023204993490967
https://x.com/Laneless_/status/2095044299033649303
https://x.com/yudhister/status/2095025704144695439

https://x.com/rasbt/status/2095141254958858496 https://x.com/gleech/status/2095048722287673443 https://arxiv.org/abs/2609.01343 https://claude.ai/chat/a97b88cc-d6d9-4bda-82dc-bd2e61ca22a3 https://arxiv.org/abs/2603.09786

Appendix: capping Astra’s number of recurrent loops#

From Pachocki, we have depth $D_{\text{Astra}} < 2 \times D_{\text{GPT4}}$.

Definitions#

$L$	the number of distinct layers in Astra
$f$	the fraction of those layers inside the looped block; $f=1$ loops the whole stack (as in “Ouro”), $f=0.5$ loops half (as in “Huginn”)
$T$	number of passes through the looped block per forward pass; $T=1$ is an ordinary transformer
$r$	$= L / D_{\text{GPT4}}$, Astra’s unrolled-once depth relative to GPT-4 (~120 layers, per the 2023 leak)
$D_{\text{Astra}}$	Astra’s effective (serial) depth per forward pass

Derivation#

The effective depth:

$$D_{\text{Astra}} = (1-f)L + TfL = L\,[1 + f(T-1)]$$

Substitute $L = r\,D_{\text{GPT4}}$ and impose Pachocki’s bound at equality:

$$r\,D_{\text{GPT4}}\,[1 + f(T-1)] = 2\,D_{\text{GPT4}} \;\;\Longrightarrow\;\; T = 1 + \frac{2/r - 1}{f}$$

$D_{\text{GPT4}}$ cancels. Our guess for $T$ now devolves to our guess for $r$ and $f$.

Result#

	$f = 1$	$f = 0.5$
$r = 1$	2	3
$r = 0.75$	2.7	4.3
$r = 0.5$	4	7

• $T \in \{2, 3, 4\}$ across most of the prior.
• “Within a factor of two” is a ceiling, so each cell is an upper bound; and $T = 1$ is excluded.

Worked example#

$r = 0.75,\; f = 0.5,\; T = 4$: 90 distinct layers, 45 of them looped four times.

$$D_{\text{Astra}} = 45 + 4 \times 45 = 225 \approx 2 \times 120$$

Cross-checks#

1. Literature regime at scale. Loopie (20B, largest public looped model) uses 2 loops; Ouro uses 4; vanilla looping degrades somewhere in 3–6 loops. 2. No-CoT horizon. If doubling the no-CoT horizon needs $\approx 1.3\times$ the layers, then $H \propto L^{k}$ with

$$k = \frac{\ln 2}{\ln 1.3} \approx 2.64, \qquad 2^{k} \approx 6.2$$

so doubling depth alone predicts $\sim 6\times$; the reported $\sim 10\times$ (assuming GPT-5.6 Sol sits near GPT-4 depth) needs the remainder from scale/RL, since looped passes are unlikely to be worth more than fresh layers. This weakly suggests Astra sits at the $T = 2\times$ ceiling rather than below it.

Caveats#

• “Depth of the computation graph” may mean serial ops rather than layers. Parallel attention/MLP blocks would break the above.
• Pachocki is probably rounding.
• The table’s spread is driven almost entirely by $r$, which is a guess.

Footnotes#

1. There are contemporary fully recurrent architectures, like Mamba, RWKV, and xLSTM, which work around this sequential bottleneck using “parallel scan” training. But these successors are still mostly not used due to other problems. ↩