How Sure Should You Be? | Lesson 2.7, Free Course

Hunches, forecasts, doubt and what's at stake.

The idea

Feeling sure and being right are different things. How sure you feel is no reliable guide to whether you're right. A hunch is worth more where things follow patterns you've had a real chance to learn. Elsewhere, it can feel just as strong and be far less reliable. A chess player's quick read of a familiar position is one thing; a hunch about a stranger is another.
Forecasting is something people can get better at. In a big contest about world events, three things went with better forecasts: training, working in teams, and tracking. Tracking meant picking out the top forecasters and putting them together, year after year. The training included asking how often similar things had happened before. A forecast like 'probably yes' can be checked once the event happens; 'who knows' can't.
Changing your mind starts with accepting you might be wrong. Some short exercises made people more open to that, for a while. One was picturing a situation the way a distant observer would see it. Most of these studies were small or medium-sized, and few have been repeated; the observer one has been. It isn't the same as having no views; it's holding them a little more loosely.
Chances are only half of a choice. The other half is how much each outcome would matter to you. An umbrella on a cloudy day is the everyday version: rain may be unlikely, but getting soaked would matter. And what matters isn't the same as money. An old argument says each extra amount of money means less, the more you already have. People still argue over whether you should always weigh choices this way.

Want the research, names and dates? They are in Go deeper, at the end.

On and off, or somewhere in between?

Picture it

Think of a belief as a dimmer switch, not an on-off switch. A dimmer can sit at 'mostly' or 'barely', which a switch can't. Evidence turns it up or down. A single surprise rarely needs to flip a belief all the way. Most of life sits somewhere in the middle of the dial.

For a child: You and a friend guess if the ice-cream van will come today. You shout 'Yes, for sure!' Your friend says 'Probably, it usually comes on sunny days.' Then clouds roll in. Your friend just guesses a bit lower. You only had 'yes' or 'no', so you have to flip all the way to 'no'. Your friend was never certain, so the clouds don't come as much of a shock.

Odds, stakes and what would change your mind.

How it applies

None of this needs maths. A rough sense of the odds is enough to leave room for a view to change. Words like 'probably' or 'a long shot' do the job. Beside the odds sits another question: how much it would matter if it happened. And there is what would make you less sure. Each part can be said in a sentence, out loud or in your head. Knowing what would change your mind turns certainty into something you can test. And with new evidence moving the view, a small surprise shifts it a little. A big one shifts it more.

Four parts · Drawing: Hacks Vitae

Forecasts, deadlines and headlines.

Where you’ll see it

At home: a forecast says rain is likely, and the whole day gets planned as if it's certain.
At work: a deadline gets called 'definitely doable' in a meeting, when it's closer to a coin toss.
Online: a single new study gets shared as the last word, as if no other study could follow it.

Signs it’s happening

Your predictions come out as 'definitely' or 'no chance', with little in between.
It's a sign of all-or-nothing thinking when new evidence changes either nothing or everything.
You can't say what would make you less sure, even when asked.

Same café, two ways to read one review.

Before and after

A new café opens nearby, and the first review you read is glowing, from someone you don't know.

The quick answer

"This place is great." One review from a stranger, and you're already certain.

The second look

"It sounds promising." Each new review can move your view up or down a little. A string of good ones turns it up; a few bad ones turn it down.

One question settled, another still moving.

A story from history: What spread the Black Death?

The Black Death struck Europe, the Middle East and North Africa from 1346. It killed an estimated third of the people across those regions.
Its cause was argued over for a long time. Then traces of the bacterium in skeletons from mass graves across Europe confirmed it was plague.
Plague comes from bacteria that pass from animals to people, usually found in small mammals and their fleas. How it spread in the Black Death was another question. The usual account blames the fleas of black rats.
In 2018 one team built models of how plague could spread. Human fleas and lice fitted the death records best in seven of nine outbreaks. The team saw this as challenging the idea that plague in Europe was mostly spread by rats. A reply that year called this only weakly supported.
In 2024 a lab study found body lice can carry the bacterium and pass it on by biting. Its authors said this supports the idea that lice may have played a role. Experts still disagree.

The cause was settled by new evidence; how it spread only shifted. Each new study moved the idea that lice spread plague a little, without settling it. That is what updating looks like: slow, and open to the next study.

Quick check3 questions · nothing is saved

1. What does the lesson say about how sure you feel?

2. In the forecasting contest, which three things went with better forecasts?

3. In the plague story, what did the lab study find about body lice?

One question worth keeping.

In one line

Holding a view as a chance, not a certainty, leaves room for new evidence to move it.

What do you feel certain about that you'd find hard to put a 'how likely' on?

Go deeperThe research, names and dates

Where this comes from

Metacognition is awareness of your own thinking, such as knowing whether you really know an answer. A word on the tip of your tongue is a common case. You know that you know it, but can't reach it yet. In a 2019 study, some people got feedback on their confidence over eight sessions. They matched their confidence to their results better than a group given feedback only on their answers. A 2022 repeat, planned in advance, found moderate evidence of no training effect.
In Mellers and colleagues' Psychological Science paper (2014), five university-based research groups competed in a geopolitical forecasting tournament. The authors found support for three drivers of accuracy: training, teaming and tracking. The training corrected cognitive biases, that is, built-in slants in judgement. It also encouraged forecasters to use reference classes: how often similar cases turned out. Tracking placed the top 2% from Year 1 in elite teams.
Researchers at Duke University defined intellectual humility as "the degree to which people recognize that their beliefs might be wrong". A 2022 review describes short exercises that raised it for a while. One was reflecting on a situation from the view of a distant observer. In another, people who first listed two things they did not know reported less confidence in their answer. Most of these experiments used small to medium samples and have not been replicated; the distant-observer effect is one exception.
The Stanford Encyclopedia of Philosophy sums up expected utility theory: "choose the act with the highest expected utility". Utility is not money: Bernoulli (1738) argued that money and other goods have diminishing marginal utility. That is, each extra amount means less, the more you have. The theory is normative, a theory of how people should decide. Whether you should always follow it is argued at length.

How likely vs how much it matters

Compared	How likely	How much it matters
The question	How likely is each outcome?	How much would each outcome matter to you?
Asks about	The chances	What's at stake for you
In the theory	Probability	Utility, which is not the same as money

Expected utility theory weighs the two together. Bernoulli (1738) argued that money and other goods have diminishing marginal utility. That is, each extra amount means less, the more you have.

More examples

Yes-or-no thoughts, and a how-likely way to say them.

If the thought is 'It will rain on the picnic', a how-likely version is 'Rain looks more likely than not'.
If the thought is 'They'll never agree to it', a how-likely version is 'It's a long shot, but not impossible'.
If the thought is 'This plan can't fail', a how-likely version is 'It will probably work, unless something specific goes wrong'.

What if…?

What if the evidence goes against something I care about?: Feelings can get in the way. In one brain-scan study, 40 people who all shared one political outlook read arguments against their views. Those views were strongly held, on politics and on other topics. Those who changed their minds more showed less activity in the insula and amygdala as they weighed the evidence. The authors said this highlights the role of emotion in resisting a change of mind. It is one study of 40 people from a single group.
What if a strong gut feeling is all I have?: In 2009 Daniel Kahneman and Gary Klein asked two things about any hunch. Is the setting predictable, and has the person had a chance to learn its patterns? They add that 'subjective experience is not a reliable indicator of judgment accuracy'.
What does it mean to be well calibrated?: Calibration is how well your confidence matches how often you are right. Your 'fairly sure' answers come true fairly often, and your 'certain' ones nearly always.

Words you may meet

metacognition: Awareness of your own thinking, such as knowing whether you really know something. Glossary →

intellectual humility: How far a person recognises that their beliefs might be wrong. Glossary →