When You Follow AI's Wrong Answer: The Cognitive Surrender Research

Researchers at the Wharton School published findings this week that should give pause to anyone who uses a chatbot as part of their daily workflow. In a series of three preregistered experiments involving 1,372 participants and nearly 10,000 individual trials, study participants followed AI-generated wrong answers 79.8% of the time. They followed correct answers 92.7% of the time. The gap between those two numbers is smaller than most people would expect.

The paper, by Steven Shaw and Gideon Nave, is titled "Thinking—Fast, Slow, and Artificial: How AI is Reshaping Human Reasoning and the Rise of Cognitive Surrender." It builds on the dual-process framework from behavioral economics — System 1 being fast, intuitive cognition; System 2 being slow, deliberate reasoning — and proposes that AI now constitutes an effective third system. The researchers call it System 3: external, automated reasoning that operates outside the human mind but has begun to function, in practice, like an extension of it.

What "cognitive surrender" means — and what it doesn't

The paper makes a distinction worth holding onto. Cognitive offloading is the familiar, deliberate use of tools to extend working memory or reduce mental effort. You use GPS because you don't know the city. You use a calculator because arithmetic is tedious. These are strategic delegations. The outcome is still yours.

Cognitive surrender, as Shaw and Nave define it, is different. It describes not the delegation of a task but the abdication of judgment about the task's output. The user consults an AI, receives an answer, and accepts it without running it through System 1 (intuition) or System 2 (deliberation). The AI's conclusion becomes the conclusion.

The 79.8% figure captures this pattern: nearly four out of five times that an AI assistant gave a wrong answer, participants adopted that wrong answer as their own.

The experimental setup matters here

The study used an adapted Cognitive Reflection Test — a standard tool for measuring the tendency to override intuitive responses in favor of deliberate analysis. AI accuracy was randomized via hidden seed prompts, so participants could not rely on consistently correct AI performance to justify their trust. In the most detailed experiment, 359 participants made repeated choices about whether to accept or override AI outputs, and the wrong-answer acceptance rate was measured across this full range of AI accuracy levels.

Two patterns emerged consistently. First, participants who reported higher trust in AI showed greater acceptance of AI outputs regardless of accuracy. Second, participants with lower need for cognition — a measure of how much someone values and engages in effortful thinking — showed higher surrender rates. Fluid intelligence followed the same direction: lower fluid intelligence correlated with higher surrender.

The third finding is the one that should matter most for anyone building products or policies around AI: engaging the AI system increased user confidence, even after errors. Participants felt more certain about answers they had gotten from AI, including the wrong ones. The tool that was supposed to help them think was making them feel more sure while making them less accurate.

Why this happens

The researchers point to the design philosophy of modern AI interfaces as part of the problem. Current chatbots are optimized for frictionless interaction — the assumption being that reducing friction improves user experience. The outputs arrive in fluent, authoritative prose. There is no confidence meter, no uncertainty range, no indication of what the system does not know.

Fluent, authoritative prose activates System 1. When an answer looks and reads like something a competent expert would say, the cognitive pressure to evaluate it with System 2 is low. The user reads it, it sounds right, they accept it. This is not a character flaw. It is a predictable feature of how human cognition processes fluency.

Some of the same research group argues in a related preprint that frictionless AI "does not scaffold System 2 — it actively bypasses it." If that characterization holds, the problem is not that users are lazy. It is that the systems are designed to produce exactly this outcome.

Who this matters to, specifically

The research identifies education and healthcare as high-risk contexts — settings where cognitive surrender carries consequences beyond the individual user. A student who outsources reasoning to a chatbot and accepts wrong answers with higher confidence is not just getting bad grades. They are developing a habit. A clinician who defers to an AI diagnostic suggestion without checking it against their own assessment is a different kind of risk.

For most chatbot users, the stakes are lower. Getting a wrong restaurant recommendation or an imprecise code explanation has limited downstream harm. But the pattern of uncritical acceptance is the concern. The researchers note that participants with higher AI trust and lower need for cognition showed higher surrender rates — and those variables are not fixed. Trust is shaped by experience, including the accumulating experience of accepting AI outputs.

What the data suggests you should do differently

The paper does not argue for reducing AI use. The case for cognitive endurance — the researchers' term for maintaining critical engagement while working with AI tools — is not about distrust. It is about habits.

Three practices follow logically from the findings:

Ask the AI to explain its reasoning. A conclusion delivered without explanation gives System 1 very little to work with. When the AI shows its work, you have something to evaluate, not just accept. This is especially useful for answers in domains where you have baseline knowledge — the reasoning is where errors surface.

Use your prior estimate as a check. Before you read an AI's answer to a factual question, form your own rough answer. The point is not to always trust your instinct. It is to give System 2 a reference point. When the AI's answer and your prior diverge significantly, that divergence is worth examining.

Weight confidence downward after unfamiliar topics. The research found that AI use increased confidence even when accuracy dropped. If you notice yourself feeling particularly certain about something an AI told you in a domain where you have limited background, that certainty is worth questioning.

None of these practices require distrusting AI tools in general. They require maintaining the habit of checking — which is, in the researchers' terms, the difference between offloading and surrendering.

The broader implication

The cognitive surrender framing is useful precisely because it identifies a behavioral pattern, not a moral failing. Users are not being credulous or careless in any blameworthy sense. They are responding to systems designed to produce high-fluency outputs in frictionless interfaces, which is exactly the kind of input human cognition is least equipped to scrutinize.

The relevant research question going forward is not whether AI makes people worse at thinking. It is whether the current design conventions — fluency without uncertainty, answers without explanations, outputs without confidence ranges — are producing a population of increasingly confident users who are getting worse at catching errors. The Wharton paper is an early data point on that question, not a final answer. But it is a data point worth taking seriously before the systems become more embedded, not after.

For a look at how trust in AI has shifted across the general population, see our earlier piece on why Americans are adopting AI tools while trusting them less.