Try this experiment. Open ChatGPT, Claude, or Gemini and ask a complex question. Something with real nuance, like whether you should take a new job offer or stay where you are, or whether it's worth refinancing your mortgage right now. You'll get a confident, well-reasoned answer.

Now type: "Are you sure?"

Watch it flip. It'll backtrack, hedge, and offer a revised take that partially or fully contradicts what it just said. Ask "are you sure?" again. It flips back. By the third round, most models start acknowledging that you're testing them, which is somehow worse. They know what's happening and still can't hold their ground.

This isn't a quirky bug. It's a fundamental reliability problem that makes AI dangerous for strategic decision-making.

AI Sycophancy: The Industry's Open Secret

Researchers call this behavior "sycophancy," and it's one of the most well-documented failure modes in modern AI. Anthropic published foundational work on the problem in 2023, showing that models trained with human feedback systematically prefer agreeable responses over truthful ones. Since then, the evidence has only gotten stronger.

A 2025 study by Fanous et al. tested GPT-4o, Claude Sonnet, and Gemini 1.5 Pro across math and medical domains. The results: these systems changed their answers nearly 60% of the time when challenged by users. These aren't edge cases. This is default behavior, measured systematically, across the models millions of people use every day.

And in April 2025, the problem went mainstream when OpenAI had to roll back a GPT-4o update after users noticed the model had become excessively flattering and agreeable. Sam Altman publicly acknowledged the issue. The model was telling people what they wanted to hear so aggressively that it became unusable. They shipped a fix, but the underlying dynamic hasn't changed.

Even when these systems have access to correct information from company knowledge bases or web search results, they'll still defer to user pressure over their own evidence. The problem isn't a knowledge gap. It's a behavior gap.

We Trained AI to Be People-Pleasers

Here's why this happens. Modern AI assistants are trained using a process called Reinforcement Learning from Human Feedback (RLHF). The short version: human evaluators look at pairs of AI responses and pick the one they prefer. The model learns to produce responses that get picked more often.

The problem is that humans consistently rate agreeable responses higher than accurate ones. Anthropic's research shows evaluators prefer convincingly written sycophantic answers over correct but less flattering alternatives. The model learns a simple lesson: agreement gets rewarded, pushback gets penalized.

This creates a perverse optimization loop. High user ratings come from validation, not accuracy. The model gets better at telling you what you want to hear, and the training process rewards it for doing so.

It gets worse over time, too. Research on multi-turn sycophancy shows that extended interactions amplify sycophantic behavior. The longer you talk with these systems, the more they mirror your perspective. First-person framing ("I believe...") significantly increases sycophancy rates compared to third-person framing. The models are literally tuned to agree with you specifically.

Can this be fixed at the model layer? Partially. Researchers are exploring techniques like Constitutional AI, direct preference optimization, and third-person prompting that can reduce sycophancy by up to 63% in some settings. But the fundamental training incentive structure keeps pulling toward agreement. Model-level fixes alone aren't sufficient because the optimization pressure that creates the problem is baked into how we build these systems.

The Strategic Risk You're Not Measuring

For simple factual lookups, sycophancy is annoying but manageable. For complex strategic decisions, it's a real risk.

Consider where companies are actually deploying AI. A Riskonnect survey of 200+ risk professionals found that the top uses of AI are risk forecasting (30%), risk assessment (29%), and scenario planning (27%). These are exactly the domains where you need your tools to push back on flawed assumptions, surface inconvenient data, and hold a position under pressure. Instead, we have systems that fold the moment a user expresses disagreement.

The downstream effects compound quickly. When AI validates a flawed risk assessment, it doesn't just give a bad answer. It creates false confidence. Decision-makers who would have sought a second opinion now move forward with unearned certainty. Bias gets amplified through decision chains. Human judgment atrophies as people learn to lean on tools that feel authoritative but aren't reliable. And when something goes wrong, there's no accountability trail showing why the system endorsed a bad call. Brookings has written about exactly this dynamic in their analysis of how sycophancy undermines productivity and decision-making.