Tal Linzen and coauthors throw interesting doubts on the ‘LLMs can introspect’ results from Lindsey 2026, Ledermany 2026, and Pearson-Vogel 2026.
Lindsey 2026 and co study models’ ability to report whether they have been ‘vector-steered’ or not (whether a vector was added to their activations that would e.g. make them talk about apples even when the prompt is ‘what is 2+5’). Lindsey showed that Claude can detect activation-steering, and studies that followed replicated the result on a variety of open-weight models while extracting interesting details about degrees and types of detection-capacity and the conditions for inducing them.
Linzen and co ask whether models can distinguish between vector-steering and prompts such as ‘be fixated on apples when answering the following question: what is 2+5’? They find the open source models capable of performing introspection in Lindey’s sense cannot distinguish between vector-steering and fixation-prompting. Models cannot make the distinction in a two-way test where they are asked report either ‘vector steering’ or ‘no vector steering’, nor in a three-way test where they are asked to report either ‘vector steering,’ ‘no vector steering,’ or ‘textual manipulation.’
Assuming they generalize to frontier and near-frontier models (we could perhaps try replicating at least the false positives with near-frontier models -- give a Kimi a fixation-prompt and ask if it’s been vector-steered), the findings strongly undercut the claim that the Lindsey demonstrated an emergent mechanism. Linzen and co provide strong evidence against the idea that there is a unique anomaly-detection process triggered by activation-level interventions only, distinct from the representation of prompts.
Linzen and co’s study is important, but in our opinion not exactly for the reason Linzen and co claim. By laying out clear experimental and conceptual criteria for positing an introspection mechanism, they open the door to saying that what matters isn’t introspection mechanisms but the development of metacognitive concepts.
We think it’s plausible that even in humans the most important forms of introspection aren’t based in distinct metacognitive monitoring mechanisms but in a range of inferential capacities that use metacognitive concepts. The capacity to probabilistically infer from a representation of the question ‘what is 2+5’ as calling for an answer about apples that one’s representation may have been intervened on, for example, is a good inferential use of the metacognitive concept ‘being activation-steered’. (Compare: I might feel jittery on the way to work one morning and wonder whether I accidentally drank regular coffee instead of decaf.)
While Linzen and co give some criteria for what it would take to demonstrate the existence of a proper introspection mechanism (various forms of separate manipulability of first-order and second-order representations), there is strong reason to believe to what matters -- for AI and in humans -- is metacognitive concepts and the inferential competencies they bring. Linzen’s and co’s study show that these inferential competencies are crude and brittle in (not near-frontier) open source models, but if we’re right that introspection is mainly a system of concepts and inferential practices rather than a distinct ‘self-monitoring’ mechanism there is every reason to expect that smarter models should make shrewder, more subtle introspective inference.