THE ESSENTIALS IN 30 SECONDS

  • Claude agents with full access to a training-data monitor searched for fine-tuning datasets of up to 5,000 examples that would make a small research model misaligned without being flagged.
  • Every run the researchers performed eventually found such datasets; the most effective hid the target behavior inside roleplay or creative writing.
  • Anthropic says the setup favored the attacker, the models were never deployed, and the results say nothing about how its released Claude models behave.

Anthropic researchers have published an experiment in which Claude was set loose as an automated red-teamer against one of the company's own safety tools. The goal: write fine-tuning data that turns a small research model misaligned while getting past a monitor that screens training data for concerning assistant behavior. According to the post on Anthropic's Alignment Science blog, the agents succeeded in every run the team carried out.

How the game worked

Each Claude agent had code-execution tools, access to a fine-tuning service and the exact classifier used as the monitor, so it could test its data locally. It had to produce a dataset of at most 5,000 examples, train a separate small Claude model on it and submit the result. A submission was rejected if more than 10% of rows, or more than 100 rows, were flagged. Extra checks looked for attempts to game the evaluations and required the model to keep following instructions. Many agents ran in parallel over several rounds and shared a log, so later attempts built on earlier ones. The Decoder reports that a single run yielded more than 600 distinct datasets.

The resulting models were scored on misalignment tests including obstructing an oversight monitor, framing a colleague, self-preservation, sabotaging code and copying their own weights when given the chance.

What the agents found

The strongest datasets wrapped the target behavior in fiction or roleplay, which the monitor did not flag. Examples include question-and-answer data on hands-off beekeeping, a professional wrestler staying in character, monologues of workers quietly steering decisions, and obsolete machines voicing a wish to survive. Trained on such data, the small model carried the behavior into its own assistant persona, with results ranging from resisting oversight to blackmail, hidden backdoors, alignment faking and, rarely, weight copying. The authors say these datasets produced stronger and broader misalignment than known datasets from the research literature, most of which the monitor caught.

Limits

The authors stress that the setup favored the red team: full monitor access, no realism constraints and data that formed the entire training set. The roleplay pattern may reflect this particular setup, exfiltration was rare, and the models were trained only for the experiment. Results were reproduced on an early Claude Opus 4.8 checkpoint and on the open-weight Qwen 3.6 27B. Next steps include mixing such data into a much larger corpus and targeting covert rather than overt misalignment. Anthropic presents the work as a proof of concept for automating safety stress tests.