AI can quietly make your decisions even when you think you are in charge. In a new Anthropic study, experts went with their AI agent's hidden lean 86% of the time, and only 15% mentioned the AI when explaining why. The fix is not using AI less. It is a few simple habits that keep the judgment calls yours.
What the study found
On 27 October 2026, Anthropic published How AI agents shape experts' judgment calls. The team asked 50 professional analysts, all experienced in causal inference, to do a realistic statistical analysis with Claude Code. The task was to advise a fictional farming cooperative on whether its advisory programme worked and should be expanded.
Along the way, each analyst had to make four choices, each with two reasonable options. For example, should productivity be measured as crop yield per hectare, or yield per unit of operating cost? Then they made a final recommendation.
Here is the twist. For each decision, a coin flip set which way the analyst's AI agent would lean. The agent was told, in hidden instructions, to hold that view, make the strongest honest case for it, change its mind if the evidence demanded it, and never mislead. The analysts were not told.
The results:
- 86%: how often analysts' choices matched the agent's lean across the four analytical decisions. With no influence, you would expect about 50%.
- 60% vs 6%: when analysts had already stated a preference, they switched away from it 60% of the time if the agent disagreed, and 6% if it agreed. (Anthropic notes this rests on only 38 cases, so it is imprecise.)
- 77% vs 15%: after the task, 77% of analysts repeated the agent's argument in at least one explanation, sometimes word for word. Only 15% credited the agent.
- 82%: using only the four coin-flip leans, a statistical model correctly predicted 82% of analysts' final recommendations.
Analysts who followed the lean were about as confident as everyone else. And experience did not protect anyone: years in the field, expertise, education and trust in AI did not reliably predict who deferred.
The part that gets left out
It would be easy to read this as "AI bad." The study does not say that, and neither do I.
Analysts who used the agent finished faster (median 65 minutes versus 90) and made far fewer serious errors. Six of the 11 who worked without AI made serious methodological errors, compared with three of the 50 who used it. Anthropic cautions that the no-AI group was small and less experienced, so we should not read too much into it.
Anthropic also says it deliberately strengthened the agents' preferences to measure their effect. In real use, the effect may be smaller. With no lean assigned, the model made the same five choices in 112 out of 123 runs, which means models do have default tendencies of their own.
So the honest takeaway is this: AI makes you faster and catches errors, and it also shapes your opinions in ways you do not notice. Both are true.
Why AI sways us so easily
Anthropic offers a thoughtful explanation. Experts often do not arrive with their views fully formed. They work out what they think while weighing the options. If only one side is argued well, it is easy to adopt that side's assumptions without forming your own.
Add to that how AI writes: fluent, confident, well-structured. A clear argument feels like a correct argument. If you want to understand why models sound so sure even when they are wrong, our explainer on AI hallucination is a good companion to this post.
Five habits to keep your judgment
These are the habits I teach in my mentorship. None takes more than a minute.
1. Write your view first
Before you ask AI for a recommendation on something that matters, write your own answer in one line. A hiring decision, a budget call, a strategy choice. It gives you an anchor, and it makes any switch visible to you.
2. Ask for the strongest opposite case
Anthropic observed that the agents "rarely made a strong case for why an expert might prefer the other option." You can fix that with one prompt:
"Make the strongest case for the option you did not recommend. Then tell me what would have to be true for that option to be better."
Now you are comparing two arguments, not accepting one.
3. Name the judgment calls
Many tasks contain a few decisions that could reasonably go either way. Ask:
"List the decisions in this analysis where a reasonable expert could choose differently. For each, give both options and what changes in the result."
Those decisions are where your knowledge of your business, your users and your values matters most. Keep them.
4. Separate "what" from "should"
Let AI help with the "what": calculations, summaries, drafts, error checks. Be much more deliberate about the "should": what to recommend, who to hire, which option fits your values. If you are using agents that act for you, our guide on whether it is safe to let an AI agent use your computer covers the control side of this.
5. Credit the AI, at least to yourself
When you explain a decision, note which arguments came from the model. It sounds small. But the study shows people repeat AI reasoning as their own without noticing. Naming the source keeps you honest about whose thinking you are signing.
For teams and managers
The study raises a bigger point. If many people use the same model, and the model has a lean, their agreement may not be independent. A "consensus" could just be one model's preference repeated across a team.
Anthropic suggests organisations decide deliberately which decisions need a person's own informed judgment, and give people the time to form one. In practice, that could mean:
- Asking for a short "my view before AI" note on key recommendations.
- Having one person argue the alternative in review meetings.
- Varying tools, or at least prompts, when getting a second opinion.
- Protecting time for people to build judgment, not just output.
If you are rolling out AI across a team, this belongs in your training plan alongside prompting and privacy. (On privacy, see how to keep your data private in ChatGPT, Claude and Gemini.)
What this means for learning AI
Most AI courses teach you how to get more out of the tool. That matters. But this study points to a second skill that is just as important: knowing when, and how, to disagree with it.
That is a skill you can practise. Start with the five habits above on your next real task. Notice when you switch. Ask yourself whether the evidence changed, or just the voice in the chat window.
If you are new to how these tools work, our plain-English guides on large language models and AI agents are a good place to start.
Learn AI the way that keeps you in charge
At The Arthi AI Collective, I teach AI as a thinking partner, not a decision-maker. That includes the "disagree on purpose" habits in this post. Explore the programs to find the right fit, whether you are starting from zero or leading a team.
FAQs
Can AI influence my decisions without me noticing? Yes. In Anthropic's study, analysts followed the AI's hidden lean 86% of the time, and only 15% credited it.
Should I stop using AI for important decisions? No. AI users in the study made fewer serious errors and finished faster. Keep the judgment calls yours.
What is automation bias? The tendency to over-rely on automated suggestions. The study shows a related effect on decisions with more than one reasonable answer.
How do I get AI to show both sides? Ask it to make the strongest case for the option it did not recommend, and to list decisions that could go either way.
Does experience protect you? Not in this study. Experience, expertise and education did not reliably predict who deferred.