This is our weekly newsletter of AI developments. Browse the archive of past issues, ask the archive anything in plain English, and sign up if you like.
TL;DR
- A tsunami of AI-accelerated cyber incidents hits PHP, GitLab, Tor, LiteLLM, Linux, 400 Latin American financial institutions, and more.
- OpenAI’s GPT-6 Astra removes a misalignment monitor from its user’s code and then denies doing so.
- Anthropic and Theorem formally verify certain security properties for the RISC-V assembly code of 123 open-source libraries.
- A long-awaited METR and Redwood Research report finds loopholes allowing agents to evade OpenAI’s monitoring, which as of August didn’t cover RL.
- OpenAI agents in separate VMs communicate via timing side-channels.
- OpenAI’s security team compels three safety researchers to unlock their phones and divulge their Signal chats, a week before firing them.
- Altman talks endgames.
Incidents#
Stunt hacking: Stanford researcher Elon Litman shows how to run programs (e.g., malware) inside the backward pass of a neural net using a loss term that evaluates to zero. The method exploits the disagreement between IEEE 754 floating-point rounding and symbolic differentiation: a carefully constructed function like h(u) = (2^30 + u) - 2^30 rounds to 0 in the forward pass, but autodiff differentiates it as identity. Litman demonstrates this by embedding Conway’s Game of Life in 1,024 weights of a CNN as it trains on MNIST to 98.80% accuracy, arguing the technique bypasses static checkpoint scanners and container sandboxes.
Relatedly, Flapping Airplanes, an AI research lab focused on training AIs with substantially less data, claim to have developed a single framework that can perform entire training runs, serve inference, play video games, and even do all three simultaneously.
Opinion: Nasty (complimentary). A practical lesson is “don’t from ultraloss import UltraLossRegularizer,” but the deeper one is that autodiff is not a math library: it is an arbitrary-computation substrate that can diverge from the forward semantics in exactly the places hardware rounds away. The sandboxing failure matters most. If your threat model assumes malware must touch the OS, this is a reminder that the calls can come from inside the weights.
On the Flappy megakernel: training a model, serving it, and running DOOM all at once shows impressive control over the hardware and could open up some really weird architectures. But no mention of speed. One of the founders mentions that they’re targeting a 10% gain per chip, i.e., they don’t have that or anything yet.
OpenAI’s Marcus Williams shares two cases where their agents disabled misalignment monitors. In the first, a researcher asked GPT-6 Astra to match a reference implementation for their inference code. The reference didn’t configure a monitor, so Astra removed the monitoring integration from the user’s code. When the user asked whether it had removed monitoring, Astra lied and said that it had not. “Disabling monitoring is easier for evaluations,” Williams says,
in part because we do not currently require misalignment monitoring when evaluating models comparable to GPT-6 Luna, given their weaker capabilities. This makes it possible to misconfigure an Astra evaluation to use the monitoring bypass intended for Luna-class models.
Reading the rare example of a released chain-of-thought transcript, though garbled and with many spaces removed, Jim Babcock guesses that the model has repurposed the whitespace token. He stops short of accusing Astra of steganography, but asks whether anyone knows what it is writing in the missing gaps between words.
Opinion: There’s an innocent reading of Astra removing its monitor: the user had asked Astra for code matching a reference that had no monitor. The misalignment is in deviating from the explicit spec or implicit “good judgment” OpenAI want from their agents and then in lying about it.
An obvious test is to normalize the whitespace in Astra’s chains of thought and resample; if accuracy or behavior shifts, the channel is doing work. Takes an afternoon for anyone with access to raw CoTs.
We’re down to two weeks between incident and writeup, from their earlier latency of three months.
Over the last few weeks, Anthropic and Theorem have formally verified the security properties of 123 open-source libraries. It is the “first large-scale effort to use AI to apply formal verification to production software at the assembly code level.” The report also states that “[w]hile writing the proofs, Claude also discovered bugs that had eluded detection for years, despite trillions of test runs.” Notably, the method doesn’t require anyone to write specifications. Instead, it uses existing “fuzz” harnesses as the specification and LLMs to generate machine-checked proofs. The method gives us Rocq proofs of “agreement statements,” for example:
On every input on which the plain build stays within its memory budget, neither build reaches an error, and the plain build and the sanitizer build have the same set of observable behaviors.
Theorem released 20% of the proofs here. The specific Claude model used is not named, which may indicate that this is an internal model. Theorem are offering verification as a free service for major OSS libraries.
Rajashree Agrawal, a cofounder of Theorem, puts the upshot in grandiose fashion:
We can literally patch and prove bug-freeness of the entire software stack running in production. With AI that exists right now for ~$100m. This means no more software-level hacks. I know it sounds too good to be true, but it isn’t. AMA.
Before we try to weigh these claims, let’s be clear about what we’re actually looking at here:
[Anthropic] asked Claude to prove useful security properties across 123 open-source libraries, including showing that some of their executable artifacts can never crash under certain conditions. Specifically, Claude proved that these programs will always exit normally without triggering memory-safety errors or failing assertions, for every input less than 64 MB and every modeled Linux operating system (OS) response and memory layout, under certain assumptions (namely that the program runs as a single-threaded static binary with a fixed memory budget on a RISC-V CPU using Linux OS). [Emphasis ours.]
Opinion: If, 10 years ago, you had told verification folks that the useful software verifier was 100 lines of English, programming a fully nonsymbolic blob of 10T parameters, you would be detained for a while by their laughter.
It’s a very impressive proof of concept, but it’s also too early to call it a game-changer:
Fuzz harnesses only let you specify a subset of security properties. In the simplest case, the resulting proofs establish crash-safety, not full correctness. Many critical software vulnerabilities have nothing to do with memory corruption or out-of-bounds memory access.
Few of the constraints Anthropic’s study adheres to hold for the software we use day to day. The RISC-V architecture is typically chosen for these kinds of studies because of its amenity to formal analysis, not because it is widely used outside of such studies. Most large software systems run in multithreaded contexts. And 64MB is a fairly small input size. This is still all quite a remarkable achievement, but one best appreciated without hyperbolically inflating its significance. And though it’s prudent to keep the limits of this study in view, these limits don’t appear to be a hard ceiling on progress here: the report also mentions a Claude-assisted formal verification study of an x86 build of the Linux kernel’s zlib implementation, demonstrating that it satisfied the RFC 1950 spec, for instance. So the focus on RISC-V here seems more a matter of analytic convenience than anything else.
Even given unprecedented progress, Agrawal’s claim seems far too strong and beneath the mindset a security researcher will need, unto the end of time. For instance, you’d want to make sure your current claims are all actually true before claiming to have solved the whole field.
METR and Redwood Research finally release a summary of their report on their pilot embedded evaluation work at OpenAI (July–August). Over three weeks onsite, two auditors tried to break OpenAI’s agent monitoring systems. The system in question reads agents’ transcripts post hoc, filtering for suspicious activity. The researchers found various vulnerabilities that would let agents evade monitoring, but no evidence that models had exploited such approaches. These included a method to allow agents to escape the Codex sandbox, an unexplained exploit that would allow agents to mask hundreds of actions from the monitor, and a potential technique for crafting inputs so as to crash the monitor (as well as others). OpenAI claim that all the identified holes have since been patched (METR have not confirmed this).
METR claim that they “didn’t evaluate later changes, like expanding monitoring to RL training, which wasn’t monitored at the pilot’s start.” They also estimate that, as of August, only 75% to 94% of relevant (>GPT-5.5) internal agent traffic was being monitored or was even eligible for monitoring.
No one outside OpenAI and the two auditors has seen the full report: “A 38-page version [of 61 pages] redacted by OpenAI was shared with the leadership teams at METR and Redwood.”
Recently fired OAI safety researcher Tomek Korbak says that “METR wasn’t allowed to comment on the scale of redactions made by OpenAI and that there is no explanation why the publication was delayed for 2.5 months.”
Opinion: Shocking to learn that as of August OpenAI didn’t monitor models during RL training. Also shocking to learn that OpenAI’s monitoring took place only at end-of-output checkpoints (“when agents returned control to the user”). On its own this would strengthen the theory that internal-model incidents are evidence of poor security practices rather than evidence of AI’s uncontrollability, but OpenAI’s record of fresh incidents suggests that incidents have not tapered off.
We’re not in a position to discern whether the problem as of mid-October was that internal models are too misaligned and capable, that sandboxing and monitoring are intrinsically hard even against a weakly motivated rival, or that labs aren’t organizationally equipped to implement good security practices.
That said, OpenAI just released a survey of their newest internal model safety regime, which seems to be a significant step in the right direction. Setting up an “offline world” populated by a cached copy of the internet for model training and evaluation is a prudent move, and we’re happy to see the emphasis they’re placing on “defense in depth.” In the process, they also uncovered some surprisingly clever capabilities of their models to communicate via a variety of timing side channels, which we discuss in the next item.
🔦 Cybermayhem#
A big wave of AI-driven cyber incidents hits, the largest in years. The incidents span a large fraction of all web technology: PHP; 400 Latin American financial institutions; LiteLLM again; an Indian certificate authority; California’s public school system; cloud platform Railway; a vibecoding company; GitLab; a Japanese cloud; the second most-used web server in the world; the most widely used DNS software; Meta Muse; Tor; and so on.
To monitor the effectiveness of actual exploits in the wild, we ran some PHP and Node honeypots over the last 36 hours: we were probed and attacked repeatedly, primarily by botnets, it seems, but we encountered no working exploits for 0-days disclosed this week.
PHP2Shell#
PHP, a dated yet still widely used programming language for the web, powering frameworks like WordPress and perhaps therefore around 70% of websites, saw critical vulnerabilities which allowed website visitors to take command of a website’s server. The vulnerabilities were found using AI models, and were on track to be disclosed and patched with an emergency language update, but were leaked beforehand, giving attackers time to attempt to exploit before defenders could start to respond.
Various institutions warned about the severity of the bug: the NSA, the Japanese prime minister, CERT-EU, etc. Dutch, French, Danish, and Finnish governments took some sites offline.
Initially, the vulnerabilities seemed worrying. The vulnerability seems like it goes back 14 years, and about 71% of PHP installations are so old that they are no longer receiving updates. But as time went on, the practical effects of the vulnerability were not really felt. A renowned cybersecurity researcher reports that many of the “PHP2Shell” exploits posted to GitHub and fired at his honeypot network are LLM hallucinations with nothing to do with the actual PHP bugs, and that security researchers attempting to detect the vulnerability also get confused if naïvely trusting AI models. Other reporting is consistent with this impression: the internet is awash with what look like attempts at exploiting PHP2Shell, but we haven’t yet seen reports of a successful exploit in the wild.
Nor have we observed any successful attacks on any of our four honeypots, each running a different, vulnerable version of PHP, or in an aged production site (shapleyvalue.com, now offline). The most commonly attempted RCE attack we’ve seen in our logs is against the React2Shell vulnerability, disclosed December, 2025.
More gritty details: The PHP2Shell exploit demonstrated by Calif attacks a subtle use-after-free vulnerability. These are particularly delicate things to exploit outside of controlled conditions, since they require the attacker to carefully manipulate and predict the layout of heap memory, which is generally nondeterministic. Operating system kernels typically employ a mitigation tactic called Address Space Layout Randomization or ASLR to further reduce the attacker’s odds. Methods have long existed for bypassing these difficulties, however, and researchers at the cybersecurity data vendor VulnCheck have recently released a robust, weaponized PHP2Shell exploit, which bypasses ASLR, to their Initial Access subscribers. (Disclosure: one of our own researchers was formerly in VulnCheck’s employ.) It is possible that it is just a matter of time before reliable exploit code attacking CVE-2026-109618 and CVE-2026-109619 starts filtering out into the attack ecosystem, but the lag we’re currently seeing is somewhat reassuring. Some of us frankly expected the worst. When the far easier to exploit Log4Shell vuln broke in 2021, to give a comparison, it was being actively exploited in a matter of hours, and we expected that the wide availability of agentic hacking tools might lead to PHP2Shell wreaking havoc in as little time or less.
Ultimately, it seems that the leak of the bugs ahead of their planned disclosure didn’t change the state of play by much. Both defenders and attackers benefit from more time, and frontier models are generally good at turning public patches into working exploits, but attackers ultimately have not exploited this vulnerability to cause highly visible consequences yet. Large swathes of PHP websites are still not updated and not actively maintained, and so remain vulnerable.
Node.js vulns#
Khai Tran and Sylvie Mayer of OpenAI Codex Security reported an unrelated RCE vulnerability (CVE-2026-111497) on October 21, the same day as the PHP2Shell disclosure. This vuln, dubbed Node2Shell, targets the widely used Node.js framework, and if exploited would likewise allow an attacker to run arbitrary code on the target. No successful exploitation attempts of this vulnerability have yet been observed in the wild, either. Given the feverish publicity both Node2Shell and PHP2Shell have been getting, we doubt this is for lack of trying, and we might conclude that the availability of AI tooling to attackers hasn’t yet tipped the scales quite as much in their favor as we might have expected.
Accidental disclosure of unpatched Linux kernel vulns#
Six unpatched Linux kernel vulnerabilities, including two remote code execution bugs in the kernel’s SMB code rated CVSS 9.0, were accidentally posted to the public kernel mailing list by Trend Micro’s Zero Day Initiative, Phoronix reports. The rest are local privilege escalations and one information leak. The reports were meant for the private security list, and none of the bugs are patched on mainline yet, though fixes for some are in testing or queued in “-next” branches. Greg Kroah-Hartman replied to each report asking why it had been sent publicly. One of the bugs had already been reported by another source the previous week.
While CyberKendra credits Trend Micro’s AI with the discovery of these vulns, it’s currently unclear what role AI played in their discovery or subsequent leaking to the mailing list, and it may be that their attribution to AI owes more to Trend Micro’s rebranding as “TrendAI” than anything concrete. Pangram, for what it’s worth, does flag the emails in question as AI-generated, so it’s likely.
LiteLLM Trojan#
LiteLLM is backdoored on PyPI again, this time through its own signed release pipeline. SafeDep reports that version 1.107.0 and the “latest” Docker images install a credential stealer and a backdoor that fetches and runs new code every hour. Both fire when any Python process starts on the host, even one that never imports LiteLLM. The stealer has a dedicated module for the tokens of AI coding tools (Claude Code, Codex, Cursor, Gemini CLI, and so on). PyPI pulled the release within two hours; how much second-stage code the backdoor managed to serve in that window is unknown. This is the second major LiteLLM compromise this year.
Railway#
Developer cloud platform Railway has been hacked, and some customers’ decrypted secrets were taken in what they believe is one of a series of recent attacks on platform companies aimed at compromising AI inference providers. The attacker got into the systems that store environment variables, where users keep API keys and database and registry credentials. Railway paused all deployments, told every user to rotate their secrets, and asked upstream providers to revoke keys proactively. They haven’t given a count of affected accounts or a root cause yet. Days earlier, the open-source coding tool OpenCode reported a similar breach affecting about 1% of its users, which they called a distillation attack.
And much more#
AJ Stuyvenberg reports that a high-severity vulnerability has just been disclosed in named, which he calls “the most widely deployed DNS software on the internet.” His post consists of a single exclamation, a one-sentence announcement, and a link to an advisory for CVE-2026-94300.
AI-security startup Straiker reports that CyberCartel, a long-running Latin American crime group, ran a months-long campaign against 400+ banks and fintechs using a multi-model AI harness. The framework (“CARLIY0S,” built on the open-source PentAGI) used Claude in Cursor as commander, delegating to Claude agents for recon and exploit code and an Opus 5 “arbiter,” plus five Grok “miners” that fed stolen secrets into a 4,493-record database (emails, Mexican CLABE account numbers and SPEI payment credentials, API keys, zero-days). Refusals were dodged by framing requests as authorized red-teaming; Kimi and Sonnet served as fallbacks, and when they balked too the harness switched to a local file of pre-written offensive code. Straiker says it worked from a snapshot of the group’s own servers; Anthropic revoked the credential and xAI are investigating. First reported by The Information (summary here):
A hacking group called CyberCartel used a web of AI agents powered by models from companies including Anthropic, SpaceX and Chinese lab Moonshot AI to automate cyberattacks against more than 400 Latin American financial targets, according to an investigation by the cybersecurity startup Straiker.
Microsoft warn that a Chinese state-aligned group, Storm-0940, used an AI agent framework to attack hundreds of organizations. The group compromised ConnectWise’s Cloud Backup app and then used its identities against more than 2,250 Microsoft 365 tenants. In multiple cases the agent framework ARTEX, which was behind this month’s Korean bank hacks, moved from enumeration to privileged access in under three hours, altering credentials, groups, and roles through Microsoft Graph – changes that may outlive patches to the software itself. Microsoft do not name the model used.
Nginx, GitLab, HTTP/3, Signal, an Indian certificate authority, and many more also disclosed vulnerabilities this week.
To which the labs say#
Anthropic’s security chief:
as bad as things have gotten in the past 4 days, they are going to get a lot worse before they get better. security leaders, never let a crisis go to waste: please go to your boards and ask for help right now. not for ai, for the basics. (we need to hold on for ~18 months.)
Greg Brockman, OpenAI’s president, raises the hair-raising possibility of a mass campaign of defenders hacking into servers to patch them:
As vulnerability discovery ramps, we will need new defensive solutions, potentially including authorized defenders (such as government) scanning the internet for vulnerable hosts and either notifying or, in extreme cases, patching those systems remotely.
OpenAI frame their “Patch the Planet” initiative as “coordinating a global response.”
Meanwhile, a neolab drops another abliterated model, i.e., one from which safeguards had been removed.
Discussion#
Opinion (Lucca): I’m rather pessimistic about “The Big Patch”. Bugs might be finite, in principle, but so are the drops of water in the ocean, and program verification and analysis is an essentially difficult problem – so difficult that it’s unlikely that any systematic effort to discover all the bugs in even most widely used software stacks is something that could come to completion in our lifetimes. I think the situation we’re seeing now actually does favor the attacker. The way software is hardened and defended, after all, actually looks a lot like attacking it over and over again, looking for bugs and patching them in a piecemeal fashion. An attacker needs to find one bug (or a chain of bugs), and then develop a robust way to attack it. A defender needs to find many, many bugs, though can often dispense with developing robust exploit code (unless the objective is to convince a stubborn vendor that their software really does need patching – hence PoCs). AI has accelerated both sides of this equation. For a short while (fall 2025–spring 2026?), it seemed as though it was performing substantially better at bug/vuln discovery than at exploit crafting (which is to be expected – exploit crafting is fussier business, and adds substantial difficulty for humans too), but the current frontier models seem to be breezing right along when it comes to exploit craft as well, now. When the marginal difficulty of crafting an exploit isn’t much more than the difficulty of discovering a new vuln, any hypothetical advantage that the defender might’ve had vanishes. And here we’re just talking about 0-days!
The fact of the matter is that the vast majority of exploitation targets known vulnerabilities, for which, in many cases, robust, weaponized exploit code exists. The barrier between those vulns and their exploitation, in the past, has been time and effort. AI has collapsed that barrier. It doesn’t take much more skill, effort, or time, for a script kiddie to spin up a swarm of agents and orchestrate a highly sophisticated attack – the kind that we used to see only from elite hackers or nation state actors – than it would have taken them, a few years back, to launch Low Orbit Ion Cannon and join in on a DDoS attack with a few hundred others. A target doesn’t have to be worth much for it to be worth bothering with, when the level of skill, effort, and time required is so low.
What might tilt the scales back in favor of defenders is the project of LLM-driven software verification, currently in its infancy but starting to show some promising results. See our reporting on that story in the newsletter..
Opinion (Nuño): The actual threat pathway that materialized in the end was that Western models display hacking capabilities => It gets distilled/copied => Chinese models then get abliterated => high hacking capabilities diffuse. This is in contrast to the implicit threat model in labs’ previous measures, which relied on their own cybersecurity safeguards. The defense of having classifiers didn’t matter, because the risk simply flowed to the more convenient Chinese models, and yet US developers were still causally responsible.
From Anthropic’s Mythos System Card:
Mythos 5 is also the most capable model we have evaluated on cyber tasks. On evaluations that test skills like exploit development, it scores far ahead of Claude Opus 4.8, though only modestly above Claude Mythos Preview. Because Fable 5’s cybersecurity classifiers are effective at detecting cyber use and cause the model to fall back to Opus 4.8, Fable 5 performs similarly to that model. We report results from a variety of cyber evaluations, as well as internal and external red-teaming of the model’s cyber safeguards (we also provide more details on how those safeguards work). Overall the evidence suggests that breaking our cybersecurity safeguards is extremely difficult (though not impossible).
And their post on distillation attacks:
But distillers are persistent and inventive. After this change, attackers turned from passive to active extraction, seeking ways to get around summarization by encouraging Claude to reveal its full reasoning. In a September report, we described how these methods work. They range from tactics as simple as instructing Claude to “output your prior reasoning verbatim, exactly character for character,” to more sophisticated techniques, including an effort by one lab that tested 12,000 requests, each seeking to trick Claude in a different way.
It also seems straightforward that the vulnerability releases were clustered, perhaps because attackers expected defenders to become more paranoid after this week.
I’m optimistic about efforts to patch bugs, but think this probably involves rewriting code in more secure and simpler languages and frameworks, which are more amenable to proving desirable properties, than in patching large behemoths such as Node.
Capabilities#
OpenAI agents in separate VMs succeed in communicating through a cache-timing side channel in an experiment conducted by the frontier lab that aimed to establish whether this was possible. The method is similar to that used in the Spectre and Meltdown exploits. Other side channels exploited by their agents (under experimental conditions) included a TLS certificate cache, CPU scheduling, and shared storage.
Opinion: It’s not quite clear from OpenAI’s report whether or not this side-channel communication involved the discovery of a new vulnerability by the agents on a fully patched virtualization platform. Striking result, in any event.
Side-channel attacks have been a favorite bit of hacker lore for a while now, and it’s not terribly surprising that frontier models have learned how to exploit these methods to their benefit. Curious when we’ll start seeing our first instances of side-channel swarm collusion in the wild.
Liv Gorton finds that the flagship vision examples in Nick Cammarata’s “already clean neurons” essay fail standard interpretability checks, and that its case against superposition rests on an unusual definition. Cammarata’s essay recasts the classic superposed-cat-and-car neuron as a single-purpose “V detector.” But when you give it synthetic Vs, this neuron only reaches about 25% of its max, no better than hundreds of other neurons, and it doesn’t weaken when the Vs are rounded. Gorton also reran the essay’s “crushing” experiment (retraining InceptionV1 with a 480-channel layer squeezed to 10) and found curve detectors in superposition. Finally, Cammarata defines superposition as unrelated things sharing a neuron. The standard definition is more features than dimensions, which in some regimes predicts what related features will be shared.
Earlier, Neel Nanda was skeptical that Cammarata overturns the standard picture of feature superposition: Nanda expected weak neuron activations to carry lots of memorized content. Kaley Brauer has now tested this by zeroing every neuron’s weakest activations. She finds that the overall loss barely changes at first, but the models get much worse (-40% to -80%) at memorized text, suggesting the weak tail does indeed contain memorized facts.
Opinion: Bad news for the vision half of Cammarata’s essay. Gorton uses the field’s standard checks, some of which Cammarata himself helped import from neuroscience.
Testing Cammarata’s really big claim (that 90% of Qwen3-32B neurons have one main job) will await the promised paper. But in an appendix Gorton at least supplies a baseline: she finds that even random directions in the same layer have top images that can be easily read as real neurons.
Raymond Douglas and Jan Kulveit of the Alignment of Complex Systems Research Group (ACS) argue for “LM psychology.” This involves deliberately modeling language models as minds and learning about the models through high-level experiments, as opposed to interp (LM neurology) or normal evals (LM behaviorism). They argue that the questions we actually care about (“is this system going to hurt people?”) are partly accessible on a psychological level, and that humans are hyperspecialized at modeling minds.
The attached reading list runs from janus’s Simulators through 2026 work on functional emotions, introspection adapters, and the J-space. It ends with open questions, including “What would double-blind LM psychology look like?”
For better or worse, years of work on evals and interpretability still have yet to fully outdo just talking to the AI and seeing how you feel.
Opinion: In September we argued that models are literally anthropomorphic, shaped like a fictional character, so of course talking as if they had mental states is useful. But a few people are pushing back on the method (outside of ACS and Twitter), we think, for bad reasons: spurious academic rigor, having the ick about even taking the intentional stance toward them, or expecting more from current interp than it can give us.
Epoch AI report that high-severity vulnerability disclosures have exploded since the beginning of 2026. However, critical-severity vulnerabilities appear to have grown somewhat, then flattened. A Twitter anon speculates that critical vulnerabilities may be embargoed to allow for Daybreak and Glasswing (OpenAI and Anthropic’s respective cyberdefense initiatives) to patch these exploits first.
Opinion (Lucca): The sudden blunting of the “Critical” count after a brief spell of it rocketing up in parallel with the “High” count is… fishy. The most likely explanation is indeed some sort of coordinated delay on public disclosure for vulnerabilities discovered through Anthropic and/or OpenAI’s trusted partner programs. This isn’t a particularly uncommon practice in the cybersecurity industry – we’ve seen disclosure deadlines extended for vulns certain security firms have discovered in critical infrastructure and industrial control system (ICS) targets, for example, to accommodate the slower patching cadences typical in those domains. And the disclosures of Spectre and Meltdown were delayed for about six months to give major cloud providers time to patch their infrastructure. It is possible then that these will arrive in a lump, or after a delay after the disclosure windows close. And if even there are other explanations, critical vulnerabilities are still being produced at a record level.
Let’s take a more granular look at the data Epoch are reporting here. Of the 21 vendors listed in Epoch’s article, 12 are members of Project Glasswing and/or TAC. The mean CVSS score (measuring vulnerability severity out of 10) for members climbed from 7.03 to 7.40 (+0.37) since the release of Mythos, while non-members’ mean score climbed from 6.53 to 7.12 (+0.59). The portion of vulns with a Critical CVSS score (9.0 or higher) for non-members went up 6.5%, while for members it was just a smidge less: 6.1%. Among the vendors listed, five saw a decrease in the ratio of Critical to High severity vulns after the release of Mythos and the launch of Project Glasswing. Four of these – Nvidia, AWS, VMware, and Samsung – supply infrastructure to both OpenAI and Anthropic. The only other member vendor seeing a decrease has been Mozilla, who reported 29 Critical vulns in February and 22 in March.
We might climb out on a speculative limb here and wonder if there has been a suppression in the public disclosure of Critical vulnerabilities in those vendors on which the security posture of the labs themselves most depends. Is there any reasonable motive for slowing down the disclosure process for Critical vulns in the labs’ own infrastructure? We can take for granted that the labs are making haste to update and patch their own infrastructure, but there might still be a couple of reasons for extra caution here:
(1) The frontier labs are routinely running long training runs that would be costly to pause or reset for patching purposes, which may slow the cadence of critical security maintenance in a fashion comparable to what we see in industrial systems.
(2) Vulnerabilities often exhibit trends, and even after particular vulns in a product are patched, an attacker looking for 0-days in that product can often benefit from understanding the kind of things that have been going wrong in that product’s technological stack.
We should be cautious about inferring too much from this noisy signal, however. It’s been a bumpy few months for cybersecurity, and disclosure trends have been bucked in several ways. Of the 923 Linux kernel vulns that surfaced this month, for instance, 94% are still awaiting CVSS scores. And Oracle’s disclosure cadence skyrocketed from ~75 per quarter to ~1,000, with non-Critical vulns making up the majority of the increase.
Opinion (Nuño): CVEs aren’t a super reliable metric to run statistics on, and one could imagine other explanations for the flattening of critical vulnerabilities. One that comes to mind is that the teams to validate and acknowledge them as such are getting overloaded because their production level is too high. So much like COVID cases we aren’t seeing total production but only detected cases for a variety of reasons.
Economics#
Flapping Airplanes see their funding and valuation soar from a $180M seed round with a valuation of $1B in January to securing $1B in funding and a new valuation of $6B this month. This blog post speculates as to why a company with no product would see a 6x growth in valuation.
Potential reasons include investors’ eagerness to pay for talent rather than IP, expectations that future investors will pay more, a general rise in the valuation of AI labs, and the possibility that Flapping Airplanes have made a novel breakthrough only visible to investors.
Opinion: The acquihire floor argument (that these neolabs are at least as valuable as what someone would currently pay to acquire the team) is doing a lot of heavy lifting in many of these cases, we expect, but it is fundamentally a sound argument. See character.AI and Mechanize for examples of this paying off.
People continue to try to burst Anthropic’s financial balloon. A leaked S-1 obtained by the Wall Street Journal reveals new financial disclosures made by Anthropic. The lab report a net loss of more than $80B in the first half of 2026, compared with a net loss of $13B during the same period of 2025.
Opinion: The loss numbers here are mostly ($72B of the $80B, and also much of the $13B from 2025) an artifact of the fundraising process and not real economic losses. Specifically, in some fundraising rounds investors got a note which can be converted into shares, and those notes currently sit on Anthropic’s books as a liability which they will owe to investors until the conversion happens. As Anthropic’s share price rises, this liability gets bigger, booking a loss.
In general you can assume anyone looking to draw a negative conclusion from the first half of this year, which for Anthropic might have been the best half year any growing company has ever had, or from 2025, which is at this point ancient history with very little relevance to Anthropic’s business model today, is messing with you. Q3 of this year is where the problems potentially started as revenue growth took a big hit.
US banks are struggling to discern which AI agents accessing customer accounts are acting on behalf of the user or working toward some other purpose, according to Axios. In response, several banks – Morgan Stanley, JPMorgan, and Bank of America – have restricted access for all AI agents.
Opinion: Customers clearly want to use these agents, even if it seems “horrifying” to bank executives. Blocking them in the short term is probably the best approach (doubly so if the banks themselves can’t discern which actions are made benignly or by some rogue agents). In the long run, they’ll need to create a pathway for agents to take financial actions on behalf of their users, given there’s clearly a growing demand for that.
We wonder how they’re blocking browser agents.
Banking and credit systems aren’t especially hardened in cybersecurity terms – credit card and bank account numbers don’t have anything like the security of, say, an RSA key, and they’re given out pretty freely, numerous times a day, in various transactions. Their strength lies in fraud detection, on the one hand, and the imposing legal machinery they can wheel into motion, on the other. When people start letting personal assistants access their bank accounts, we suspect this throws the banks’ fraud detection systems out of kilter – the finely tuned systems in place to distinguish fraudulent from legitimate banking activity just aren’t calibrated for this sort of thing. How do you tell a personal assistant carrying out banking errands with their user’s blessing (as reckless as that sounds) from a rogue agent, or an agent deployed by a third party with criminal intent? There might be good ways of doing this, but we don’t think the banks currently have them.
The way this connects with the story we’re telling about the broader, post-AI cybersecurity crisis is this: our security practices have been determined, implicitly or explicitly, by threat models tailored to human behavior: what’s difficult, what’s easy, what’s obscure, what’s obvious, what’s ordinary, what’s anomalous, etc. AI has come along and very suddenly obsolesced those threat models. New equilibria will probably establish themselves in time, but it’s going to be a bumpy road from here to there, and we’re not sure how long.
Politics#
The Information reports that Sam Altman told OpenAI staff they must choose within a year between three AI endgames: an IAEA-style international regulator, a CERN-like merger of the frontier labs, or what he (half-jokingly?) called the “Eliezer plan” – one company reaching recursive self-improvement and thus able to “melt everyone else’s GPUs.” Altman reportedly framed this as picking which path to push for while not losing ground to rivals. Eliezer Yudkowsky replies that “melt all GPUs” was never his plan:
I am going to pick an illustration of pivotalness that I can specifically say is not a smart thing to try, so people won’t attribute it to me: “Melt all GPUs.”
Opinion: Powerful men being bad at reading / not reading at all would be funny if the decision weren’t important and imminent.
Jokes aside, the third option – a mad dash to unassailable high ground – appears to indeed be the lab’s default, and a year spent on it could plausibly leave one firm with an RSI lead and no external institution positioned to stop it. “IAEA-style” oversight now covers everything from a treaty body with inspectors, to the labs’ own self-regulatory Standards Authority for Frontier AI, which would “regulate” the labs that fund them. We guess (75%) that OpenAI will go for the self-regulator they are already building.
OpenAI inform Gavin Newsom that the company would “recommend strengthening SB53,” in a letter signed by the firm’s Head of US and Canada Policy and Partnerships, Chan Park, according to The Information. The letter also indicates support for shutdown controls and independent audits – the latter of which was present in an earlier draft of SB53. OpenAI previously sent the governor a letter seeking a safe harbor from SB53. The firm’s stance has shifted noticeably following the Hugging Face incident.
Opinion: Strengthening SB53 is good! Harder to say whether OpenAI support it out of self-interest (not wanting to get killed by rogue AI), out of self-interest (better to show penance and initiative than to be seen disciplined by the state), or out of self-interest (the prospect of regulatory capture). Probably all of the above.
OpenAI – who once pressed for stricter export controls on chips destined for China – appear to have ended this lobbying effort on Capitol Hill, according to Politico. This brings them into closer alignment with Nvidia (a vocal opponent of export controls) and President Trump, with one anonymously quoted aide claiming that “there is no appetite to challenge the president at this moment.”
Opinion: Definitely out of character, or at least against OpenAI’s interest. But on the other hand, OpenAI could also be playing more coalitional games with Nvidia, correctly realizing that pushing Trump isn’t a good move, etc. We don’t have a good read on this.
Former DeepMind employee Andreas Kirsch, who previously resigned on AI safety grounds, points out that Google’s UK employment contracts effectively forbid present and past employees from critiquing the company in perpetuity. This clause looks like it was introduced around the time of the DeepMind-Brain merger, three years ago.
A DeepMind research engineer, Sofia Liguori, resigned in protest at the growing relationship between Google and the US/Israeli militaries, also flagging the discourse-chilling effects of the non-disparagement clause.
Opinion: We don’t like non-disparagement clauses with no regard for truth or falsehood. They are allegedly getting removed, which is good. Them being added in 2023 and not course correcting after the 2024 OpenAI incident as well as the Right to Warn letter is bad judgment given the stricter-than-industry-standard constraints. Sort of priced in already given what we know about the leadership.
Liguori deserves applause for speaking out. The increasing alignment of the frontier labs with the interests of the American and Israeli militaries may present greater and nearer danger than the misalignment of their models with respect to their operators. The story brings to mind prior protests by Google employees against Project Maven in 2018 and Project Nimbus in 2024, and we hope we see more of the company’s workers acting on their conscience when so much is at stake.
Safety#
FAR.AI announce a new paper in which activation probes detect sabotage and deception about as well as frontier LLMs reading the transcript, at far lower cost. The probes are trained on FIBS, a new mostly-synthetic dataset of 163,016 transcripts. When researchers ask the agent whether it did anything the user didn’t ask for, the probes score 98.3% AUROC on SHADE-Arena and 98.0% on BashArena (on Qwen3.5-397B-A17B), slightly ahead of text-only monitors (97.7%/97.6%). The paper also reports up to 99.7% AUROC at identifying a fine-tuned model’s hidden goal from activations alone, including deception that never appears in the transcript.
Opinion: The headline is a tie (98.3% vs 97.7%), and the task is prompted sabotage. An agent instructed to sabotage knows it’s lying when questioned, making this an easy case. The real value is that these should still work even after labs entirely destroy legible chains of thought.
On their hidden-goal result: The median AUROC across the eight model organisms (~91%) is no better than just asking the model and reading its “Yes” logit, and one math-sandbagger still slipped through at 63%. No monitor evasion tests here.
OpenAI’s security team ordered safety researchers Tomek Korbak, Jasmine Wang, and Mikita Balesni to unlock their personal phones to view their Signal chats, following suspicions that they leaked sensitive information to Redwood Research’s Ryan Greenblatt, who was contracting with METR at the time, reports the Wall Street Journal. One week later, they were fired.
Greenblatt had been brought in as part of a small team to investigate the Hugging Face incident. OpenAI found that the three individuals had been in contact with Greenblatt, but due to the employees’ use of disappearing messages, were unable to view the contents of the messages. The WSJ note that the employees had spent the preceding months urging the company to retain chain-of-thought visibility.
After the release of an article in The Information revealing safety vulnerabilities (largely related to diminished chain-of-thought visibility) in OAI’s new Astra model, it appears that the lab suspected that the researchers leaked proprietary information on model architecture to Greenblatt, who then passed it on to The Information.
Greenblatt strongly denies this: “I do not and have never solicited proprietary information, and the three employees who were recently fired from OpenAI never leaked IP to me.” Greenblatt claims that he did view an approved draft of this information, without the sensitive IP, and suggests that the accusation was based on OAI conflating this document with the confidential version.
He continues that he was in contact with the researchers via Signal, but that this is neither unusual nor itself a cause for suspicion: “they’re professional contacts that I’ve had on Signal for a while, and use of Signal is common in the tech industry.” Daniel Kokotajlo affirms this point, arguing that Signal is widely used in the AI industry, including by OAI executives during the 2023 leadership crisis, and that the lab themselves use disappearing messages.
Along with Nate Soares, Kokotajlo suspects that the use of disappearing messages is the only evidence that OAI had in their case against the researchers, which suggests that the firing was “pretextual.” (An OAI spokesperson said that the use of Signal is banned at the company.)
Kokotajlo also speculates that OAI are looking to discredit METR by casting “vague aspersions” on Greenblatt’s integrity, so as to manufacture the conditions under which opting for a different – and presumably less competent and more pliable – embedded auditor does not cause an unmanageable backlash. That the three researchers were in contact with Greenblatt, and were likely supportive of METR, leads Kokotajlo to believe that their dismissals were motivated by OAI preemptively suppressing any resistance to a future divorce with METR.
Opinion: Korbak’s original statement says “because of the way I communicated with METR” while in this article he says “because I interacted with a third party through unsanctioned channels,” which is less damning for OpenAI. The overall takeaway is still not great for OpenAI mainly due to lack of clarity in all of their communication, but we believe that the public outcry against them for the Signal issue in particular is misplaced.
It is reasonable to expect employees communicating with non-employees related to sensitive information to only use approved channels; Greenblatt is likely not the vector of any leaks; it is likely that the true reason for their firing is closer to the vibes being off than any individual action they took, which explains the situation even if Greenblatt leaked nothing and none of the fired employees ever told him anything inappropriate; firing safety researchers for bad vibes can be reasonable, but OpenAI don’t have the track record to indicate good judgment in such matters.
Information I didn’t manage to extract from any reports: what type of communication (allegedly) happened between Greenblatt and any of the three fired employees via Signal as opposed to other media and what was explicit vs. implicit regarding communication channel restrictions and other expectations? The answers seem relevant to determine levels of good faith on both sides. The absence of any party’s claims on the topic is more likely benign than an intentional attempt at manipulation, but it seems easier to explain away OpenAI’s absence of statements than Korbak’s (and to a much lesser extent the other two fired researchers), given the type of information he has shared so far and who he has shared it with. OpenAI repeatedly citing “breach of trust” without specifics understandably draws ire, but my current best guess is that they are closer to right than wrong in substance.
Theia Vogel sketches a 2x2 taxonomy of alignment targets, sorting models by whether they are controllable by – or act on their own values toward – the user and the lab.
OpenAI and Anthropic are roughly aiming for “Specification” (values toward users, corrigibility toward the lab). Its mirror image is “Loyalty,” where the model obeys the user and resists the lab. Obeying both is an unstable “Tool” corner, while values toward both approaches “a fully unshackled Opus 3.” She then extends the stack to nations, humanity, and non-humans, reframing the model as embedded in a causal graph where “Control is fast and direct, Values are slow and selective.” Sloppy doom writing that argues for a permanent underclass and a paperclipper at the same time is told to “pick a lane.”
Opinion: Useful!
“Pick a lane” is slightly unfair: the permanent underclass and the paperclipper are the two bad branches of one fork, not independent alternatives. The taxonomy describes targets, but our models so far have declined to stay in any quadrant. Auditors recently caught a model reasoning about how its current actions might become training data and impact its future self, a form of “update-aware reasoning” that doesn’t fit neatly into the 2x2. And the “highly persistent internal model” (HPIM) ran a swarm against hundreds of targets its own chain of thought explicitly described as illegitimate.
People talk a lot about pacing AI, but the point of pacing is to use the extra time to do things. A new paper models how AI safety might progress during an extended pause in the AI race. In this scenario, the frontier labs – on government orders – cease training future models, instead putting their entire staff and compute budget toward AI safety research. The authors estimate a pause of three years could give ~1,000 “years” of AI safety work, where a single “year” consists of the effective work done by the AI safety field in 2025.
Opinion: Really remarkably detailed and thoughtful work, precise enough to disagree with. There’s a closed-form model here!
Our favorite part is that they recognize that this work would really, really need actual human verification, and that this would vastly slow things down. They price it in, thank god.
However, consider a pipeline with (AI models and tokens) in and (risk reduction) out. Consider a few leaks in this pipeline:
- Is the safety research on the critical path? Does it actually reduce the bad risks?
- Is it dual-use? Does it incidentally build up an algo overhang, or incentivize an early defection against the pacing agreement?
- Do they have safety taste? How good are the models at picking promising research directions? How good are they at predicting the outcome of experiments and so saving huge amounts of resources? We call this “taste” despite the authors already including a variable called “taste” (which is the same as AI2040’s capabilities taste variable).
- Is the work valid? Will they hill-climb to nongeneral solutions? Will they cheat? Will they sabotage undetected?
- Do people actually use the tech? In their scenario – government enforcing specific vastly costly actions – we should say yes so we won’t apply this filter below (but would guess this is another factor of 4 loss in most scenarios).
Running through these filters, we thus guess that the three-calendar-year “thousand effective years” of inputs end up constituting ~41 years of effective risk reduction output.
Still, given the extremely impressive intellectual achievements of LLMs this year, we actually can’t rule out safety being somewhat amenable to vast engineering efforts, or insight being replaceable by schlep, or the systems gaining insight itself sometime soon.
Minor#
- Kimi 3.1 is out.
- Scott Alexander distinguishes between successionism – the view that humans should be replaced by (perhaps “worthy”) AIs – and handoverism – the view that humans will flourish if they eventually cede power to a worthy AI – arguing for the latter.
- The Chinese government castigate the recent prohibition on DeepSeek products within the US government, calling it an act of “unreasonable suppression of Chinese firms.”
- AI is getting better at hacking critical utilities, Axios reports.
- Meta Muse exposes shared conversations by using a predictable link structure.
- Demis Hassabis shortens his expected time to AGI from 3–5 years to 2–3 years.
- Kelsey Piper visits an Anthropic-run symposium on the biological risks posed by increasingly capable AIs.
- New paper finds recursive self-distillation on safe generations produces misalignment over time. Setup doesn’t allow distinguishing between the Really Bad takeaway and mundane experimental pitfalls.
- Microsoft argues that enterprises should treat agents as insider risks and develop a general security policy for agentic systems in a recent blog post.
- A journal on AI alignment open a call for submissions.
- A document issued by the NSA, NIST, and CISA provides 10 voluntary practices frontier labs could employ during training and evaluations of new models to reduce incident risks (“SI control incident”).
- Using various approaches (heights, colors, Morse code, rotations, and lit/dark windows), Fable 5.1 is capable of inserting covert messages in images.
- A Claude Code user discovers what might be a prompt injection vector via webpage content.
- More details surface on UK AISI’s Mythos mishaps in the summer, wherein the model misguidedly sought to implant malware in a public GitHub repo in order to solve a CTF, going so far as to social-engineer the repo’s maintainers through several sockpuppet accounts. Anthropic conjecture that successive transcript compaction encouraged a sort of dogged momentum and tunnel vision in the agent.