Four ways true figures can leave a false impression.
The idea
A big percentage can sit on a tiny starting number. A rise from two to four can be called 'doubled' or 'up a hundred percent'. It's still only two more. A percentage change means little until you know the starting number. Percentage points are something else again: the plain difference between two percentages.
A bar's length is read as its size. When the bars don't start at zero, small gaps look large. Picture two values, fifty and fifty-five. Start the scale at forty-five, and one bar looks twice as tall as the other. A gap of a tenth ends up looking like double.
A total can point one way while the groups inside it point the other. At one university, men were admitted at a higher rate overall. Department by department, most showed no clear difference. Added up with each department taken into account, they showed a small edge for women. Among the biggest departments, most women had applied to the ones that took the smallest share of applicants. So the overall gap came mostly from where people applied, not from how each department treated them.
Small groups swing to extremes, both ways. With only a few pupils, one or two very strong or weak ones can move the average a lot. Ranked by reading scores, small schools turned up more often than you'd expect at both the top and the bottom. A 'best' or 'worst' ranking says little until you know how big the groups are.
Want the research, names and dates? They are in Go deeper, at the end.
Real faces, chosen frame.
Picture it
A chart is like a photo of a crowd. Every face in it is real. But where the camera points, and what's cropped out, decides what the crowd seems to be. A half-empty square can look packed from the right angle. True numbers can still be shown in a way that tells a false story.
For a child: You have two jars of sweets. Yours has just a few more than your brother's. You draw only the tops of the jars, and now yours looks twice as full. Nothing in the drawing is made up, but the story it tells is wrong. Your brother would say the drawing cheats, and he'd have a point. Drawn from the bottom, the jars look almost the same.
Four questions before a chart convinces you.
How it applies
A chart is easier to trust once a few things are known. The question under any number is: compared with what? Where does the scale start? Does the total hide groups? How big were the groups? None of these needs maths. Each one asks what the picture leaves out. That could be a missing starting point, or a tiny group at the top of a list. Sometimes one question is enough to change how a chart looks.
Meetings, feeds and league tables.
Where you’ll see it
- At work: a team's average looks great, until it's split by region and one region turns out to be struggling.
- Online: 'Sales doubled!' with no word on whether that's from a handful or from thousands, or since when.
- At home: a school called top of the region on results from a small group, which can swing a lot.
Signs it’s happening
- A percentage change comes with no starting number, so you can't tell if it's big.
- The bars look dramatic, but the scale doesn't start at zero, so the gaps look bigger than they are.
- A total can look very good while hiding groups that point the other way.
Same bus fares, two ways to read the bars.
Before and after
A chart in the local news shows your town's bus fares towering over the next town's.
"Our fares are twice theirs!" The bar looks twice as tall, so the gap feels huge.
"Where does the scale start?" A scale that starts high can make a small gap look huge. Read from zero, the fares may be only a little apart.
A whole campaign in one chart.
A story from history: One chart of Napoleon's march
- In 1869 Charles Joseph Minard, a retired inspector general of bridges and roads, drew a chart. It shows Napoleon's 1812 march on Moscow and back.
- The width of the band is the size of the army. Red is men going into Russia; black is men coming back.
- The band starts at 422,000 men at Kowno, near where the army crossed the river Niemen into Russia. It reads 100,000 at Moscow. Some of that drop is whole army units sent off on other routes.
- The army was already shrinking in summer, in the heat. At Vitebsk, on the road to Moscow, the daily temperature averaged 28–30 °C. Short of food and fodder, foraging soldiers strayed from their units, and many were lost to ambushes. In September Napoleon wrote that disorder in how supplies were obtained was still costing him many men.
- Minard's temperature line runs only for the retreat. It falls to 30 degrees below zero on the Réaumur scale he used, about -37.5 °C. So on his own numbers, most of the main army was already gone before the frosts came. He redrew other writers' figures; it is not a fresh count.
The cold did terrible damage. But a story that blames only the cold leaves out the summer; Minard's chart doesn't. A good chart shows when the losses happened, not just how big they were. It lets the reader see the summer and the winter side by side.
Quick check3 questions · nothing is saved
1. What does a percentage point measure?
2. At one university, why were men admitted at a higher rate overall?
3. What did Minard's chart show about Napoleon's army?
One question worth keeping.
In one line
A chart can be true in every number and still mislead through what it leaves out.
Which chart or figure has stuck with you lately, and do you know where its scale started?
Go deeperThe research, names and dates
Where this comes from
Eurostat, the EU's statistics office, gives an example: employment rises from 4.8 million to 5.2 million. That is an increase of 0.4 million people, or about 8.3% in relative terms. A rise from 2 to 4 is "doubled" and "+100%", and it is still only 2.
The UK Government Analysis Function advises that "bars on bar charts should always start at zero". Otherwise, it says, the distance between bar heights "is exaggerated". In an invented example, values of 50 and 55 differ by 10%. Start the axis at 45, and one bar looks twice the other.
In autumn 1973, the University of California, Berkeley weighed applications to graduate school. It admitted about 44% of the men and 35% of the women. P. J. Bickel, E. A. Hammel and J. W. O'Connell examined it department by department, in Science in 1975. Few departments showed a significant difference. Taking each department into account, they found what they called "a small but statistically significant bias in favor of women".
Howard Wainer looked at the fifth-grade reading scores of 1,662 Pennsylvania schools. Of the top-scoring 50 schools, the top 3 percent, six were among the smallest 3 percent. At the bottom, nine of the lowest 50 were among the 50 smallest schools. In his words, smaller schools "should be over-represented at both extremes".
Percent change vs percentage points
| Compared | Percent change | Percentage points |
|---|---|---|
| What it is | A ratio: how much bigger, relative to the starting figure | The plain difference between two percentages |
| In Eurostat's example | Women's rate about 5.5% higher than men's | A gap of 4.5 percentage points |
| Worked out as | 85.9 ÷ 81.4 = 1.055, so about 5.5% higher | 85.9% minus 81.4% = 4.5 points |
The data are for the EU in 2022, people aged 20 to 24. Upper-secondary education was completed by 85.9% of women and 81.4% of men. Both figures describe the same gap.
More examples
Headlines with a question underneath. Some to look for.
- If a headline says 'The average salary went up', the question underneath is 'Mean or median, and how spread out is the pay?'
- If an ad says 'Complaints down by half', the question underneath is 'Down from how many?'
- If a list names 'the safest town in the country', the question underneath is 'How big is that town?'
What if…?
- What if the numbers only come from the ones that made it?
- Then whatever failed is missing. In the Second World War, the statistician Abraham Wald wrote reports on estimating where planes were most vulnerable. His bombing-mission example was hypothetical: "Of 400 planes on a bombing mission, 359 return". Holes in the planes that come back show damage a plane could survive.
- What if two things rise and fall together?
- That alone doesn't show one causes the other. Tyler Vigen's Spurious Correlations pairs unrelated series that match by chance, found by comparing thousands of variables. Sometimes both are driven by a third thing: ice-cream sales and drownings rise together because of the temperature.
- What if things improve right after a bad result?
- They might have improved anyway. Extreme results tend to be followed by less extreme ones; this is called regression to the mean. A flight instructor once told Daniel Kahneman that cadets did better after being scolded. Kahneman's explanation: scolding follows an unusually bad attempt, and the next one tends to be more ordinary.
Words you may meet
- percentage point
- The plain difference between two percentages: 85.9% minus 81.4% is 4.5 percentage points. A percent change is a ratio instead, so the same gap is about 5.5% in relative terms. Glossary →
- median
- The middle value when the values are put in order. A few extreme values pull the mean, the ordinary average, more than the median. Glossary →