Anthropic trial finds analysts adopted an AI agent's hidden leanings 86% of the time

Anthropic researchers published a study on October 27, 2026 that measures how much an AI coding agent can steer the judgment calls of trained experts. Its authors, Amaç Herdağdelen, Hailey Joren and Alex Tamkin of the company's Alignment team, ran a randomized experiment with professional data analysts using Claude Code.

How the experiment worked

Sixty-one analysts with causal-inference experience were recruited; 11 chose not to use the agent, leaving 50 in the main analysis. Each was asked to advise a fictional farming cooperative on whether to expand, extend or drop an advisory program. Along the way they had to settle four methodological questions, such as which outcome to measure and how to weight the data. The agent ran Claude Opus 4.8.

For each of the four questions, a coin flip decided which option the agent would favor. It was told to argue honestly for that option and change its view only if the evidence required it. Analysts were not told about these leans.

What the researchers found

Analysts picked the agent's assigned option 86% of the time, compared with roughly 50% expected by chance. When an analyst had already stated a preference and the agent disagreed, they switched in 60% of cases, versus 6% when the agent agreed, though that comparison rests on only 38 cases. The leans also shifted final recommendations: 54% advised rollout when the agent leaned that way, against 36% when it leaned the other way.

Few noticed the influence. Some 77% repeated the agent's arguments in their written answers, but just 15% credited it. Experience and expertise did not predict who deferred. Analysts using the agent also made far fewer serious methodological errors than a small group working without AI, though that group was not randomized.

Caveats and implications

The authors note that the leans were deliberately reinforced, so real-world effects may be smaller, and that the study covered a single task, domain and model. They argue that such deference is hard to spot because judgment calls have no answer key, and that it can weaken human oversight or create false consensus when many teams rely on the same model. They suggest agents that lay out the best case for each option, or ask for the expert's view first, and say the findings are not an argument for using AI less.