Cognitive Biases, Checked: 14 Famous Findings and Whether They Held Up

Cognitive biases are often presented as settled science. They are not all equally settled.

Psychologists have re-run many of their field’s most famous experiments — often with larger samples, in many labs at once, and sometimes with the analysis plan registered in advance. At least one came back stronger than first reported. Several came back smaller. A few have not come back convincingly at all.

This page checks fourteen of them. The cheat sheet on this page (“What we found”) has a verdict and the key study for each; the sections below explain how to read those verdicts.

01The verdicts

All ◐Partly✓Holds up⇄Still argued?Can't confirm

01 Anchoring: a number mentioned in a question sways the answer ✓Holds up Held up. Described by Tversky and Kahneman in 1974.

02 Hindsight bias: after the fact, outcomes seem more predictable than they were ✓Holds up Held up. A 2021 close replication of Fischhoff's 1975 study 'found support for hindsight bias in retrospective judgments'.

03 Confirmation bias: we judge evidence more kindly when it suits us ◐Partly Real, but it is a family of effects, not one finding.

04 Framing: the same choice described as gains or losses changes decisions ◐Partly Held up, at about half the original size.

05 Sunk cost: people keep investing because of what they have already spent ◐Partly Replicated as a small effect in a hypothetical scenario.

06 The marshmallow test: children who wait for a second treat do better in life ◐Partly Smaller than claimed.

07 The bystander effect: nobody helps when others are around ◐Partly Real, but usually misread.

08 Power posing: standing like a superhero changes your hormones and behaviour ◐Partly The hormone and behaviour claims failed; the feeling replicated in that study.

09 Facial feedback: holding a pen in your teeth to force a smile makes cartoons funnier ◐Partly The pen study did not replicate; a small effect of facial feedback seems real.

10 Loss aversion: losses hurt more than equal gains ⇄Still argued Genuinely contested. A 2018 review argued that current evidence 'does not support that losses, on balance, tend to be any more impactful than gains'.

11 The Dunning-Kruger effect: the least skilled are the most overconfident ⇄Still argued The overestimation is real; its size and cause are disputed.

12 Stereotype threat: reminding girls of a stereotype lowers their maths scores ⇄Still argued Probably smaller than claimed.

13 Ego depletion: willpower is a resource that runs out when you use it ⇄Still argued The strong version failed; a very small effect is disputed.

14 Elderly priming: reading words about old age makes people walk more slowly ?Can't confirm The original finding has not been confirmed.

6 of 14 shown · a mix of every verdict Show all 14 ▾

02Why this matters: the replication projects

In 2015, the Open Science Collaboration published an attempt to repeat 100 studies from three psychology journals. By one of its measures, “Thirty-six percent of replications had statistically significant results”, and the replication effects “were half the magnitude of original effects, representing a substantial decline.”

It is tempting to read that as “only 36% of psychology replicates”. That goes too far. The project used several measures of success, and 36% is only one of them. A fairer summary is narrower: many published effects were weaker than first reported, and some did not reappear.

The Many Labs projects took a different approach — testing a set of well-known effects across dozens of labs at once. In the second of them, covering 28 findings, the authors reported that 15, or 54%, “provided evidence of a statistically significant effect in the same direction as the original finding.”

100	Studies from three psychology journals the 2015 project tried to repeat
36%	Replications with significant results, by one of several measures
Half	The size of the replication effects against the originals
54%	Many Labs 2, covering 28 findings: 15 showed a significant effect in the original direction

Many published effects were weaker than first reported, and some did not reappear.

03How to read the verdicts

• Holds up — the finding replicated in large, independent tests.
• Partly — something real was found, but it is smaller, narrower, or different from the popular version.
• Still argued — credible studies point in different directions. We say which way the evidence currently leans and why.
• Can't confirm — the original finding has not been confirmed by a convincing replication.

Two cautions apply to every one of them. A single failed replication is not a disproof: it may have tested the idea differently, or been unlucky. And a single success is not the end of the argument either. What matters is the weight of evidence across several good tests — which is also why these verdicts could change.

04What the fourteen show — our reading

The biases about judgement mostly held up, sometimes smaller. Anchoring and hindsight bias replicated clearly; framing replicated at about half size; the sunk cost effect replicated as a small effect in a hypothetical scenario.

The findings about subtle manipulations mostly shrank or failed. Ego depletion, elderly priming, power posing’s hormone effects and the pen-in-mouth smile all claimed that a small manipulation changes how people feel, perform or act. Those are the claims that struggled most when tested again.

Several famous findings are real but routinely misdescribed. The bystander effect is real for each individual, yet in real conflicts caught on camera someone usually helps. The marshmallow test found something, but much less than the popular story. The Dunning–Kruger effect has a real core, but how much of its classic pattern is real is still argued over.

Judgement biases	Mostly held up, sometimes smaller: anchoring, hindsight, framing, sunk cost
Subtle manipulations	Mostly shrank or failed: ego depletion, elderly priming, power posing’s hormone effects, the pen-in-mouth smile
Misdescribed	Real but routinely misdescribed: the bystander effect, the marshmallow test, Dunning–Kruger

Those are the claims that struggled most when tested again.

05Using this without over-correcting

It is tempting to swing from “psychology proves everything” to “psychology proves nothing”. Neither is right. The replication projects are themselves psychology working as it should — checking its own claims and changing its mind.

The practical rule for any study you hear about: ask whether it has been independently repeated, and how big the effect was the second time.

For the method of checking a study yourself, see how to check something online and our guide to numbers that mislead. The Dunning–Kruger chart is covered in Older Than You Think, Part 2.

How we checked this · Last checked September 2026

• Each finding was checked against the original studies and the large replication projects and meta-analyses listed below.

Checked September 2026.

How we check every article →

06Sources