OpenAI says activation probes have screened every assistant completion since GPT-5.6, token by token

OpenAI has published a methods note explaining how it uses activation probes, small classifiers that read a model's internal state rather than its text, to police what its assistants write. The post, released on November 2 on the company's alignment research blog by five researchers, says the probes have run on all assistant completions of its most capable models starting with GPT-5.6. It follows an October 29 post in which OpenAI presented probes as the first, deliberately sensitive layer of its system for spotting misaligned behavior.

How the probes work in real time

The probes take residual-stream activations, usually from a layer near the middle of the network, and return one score per sampled token while the answer is being streamed, computed directly on the inference engine. If a score crosses a threshold, three things happen: streaming to the user stops, any tool calls are blocked until they are marked safe, and the case is escalated to a reasoning model that makes the final judgment. OpenAI describes them as one layer of a defense-in-depth stack that also includes text-based classifiers.

The chosen design is an attention-style probe combined with a moving average, which the authors say keeps the memory needed during streaming small. Training uses up to about one million messages, mixing synthetic and de-identified conversations, with labels produced by the company's strongest reasoning models acting as judges. Because results vary a lot between training runs, the team tries at least 32 random seeds per configuration and attributes roughly 85% of the performance spread to initialization alone.

What OpenAI claims, and what it does not

In OpenAI's own red-teaming, probes were overall more robust to jailbreaks than content-based classifiers, particularly against attacks not specifically optimized to fool them. The authors caution that real attackers can adapt to the full safety stack. They also report that probes transfer well across post-training checkpoints but less well across pre-training checkpoints, and that their performance keeps improving on larger frontier models.

The limits are stated plainly. Probes are supervised, so they detect only the behaviors they were trained on, and because they are tied to a specific model's activations they must be retrained for each new base model. The post gives no recall or false-alarm figures for deployed systems and warns that its charts come from internal experiments that are not necessarily representative of production classifiers. For users, that means the practical effect, such as how often a legitimate answer is cut off, remains unknown from the outside.