Numbers · 3 min read
Base Rate Neglect: Why a Positive Test Often Isn't What It Seems
A test that's 99% accurate says you have a rare disease. Are you 99% likely to have it? Usually far from it. Base rate neglect, explained with whole numbers.
By the Cognosc team ·
Here’s one of the most important probability questions most people get wrong, including many doctors:
A disease affects 1 in 1,000 people. A test for it catches 99% of people who have it, and wrongly flags 1% of people who don’t. You test positive. How likely is it that you have the disease?
Most people say about 99%. The real answer is about 9%.
Try it
Who really has it?
A thousand people take the test. Change how rare the condition is, and how often the test wrongly flags someone healthy.
11 people test positive. Only 1 of them have the condition: 9% of positives are real. The rest are false alarms from the 999 healthy people.
Work it out with people, not percentages
The easiest way to see it is to picture a crowd. Imagine testing 1,000 people.
- People who have the disease: about 1 in 1,000, so 1 person. The test catches 99% of cases, so they almost certainly test positive.
- People who don’t: the other 999. The test wrongly flags 1% of them, so about 10 people test positive even though they’re healthy.
That’s about 11 positive results, and only 1 of them is real. So if you test positive, your chance of having the disease is about 1 in 11: roughly 9%.
What went wrong
The mistake is called base rate neglect: focusing on the test’s accuracy and ignoring how common the condition was to begin with (the base rate).
When a condition is rare, even a small false-alarm rate, applied to the huge number of healthy people, produces more false alarms than there are real cases. The test isn’t bad. The condition is just rare.
The two numbers people mix up
The trap is confusing two different questions:
- If you have the disease, how likely is a positive test? That’s 99%. It’s what the test’s accuracy tells you.
- If you test positive, how likely is it that you have the disease? That’s about 9%. It’s what you actually want to know.
They sound almost the same, but the second depends heavily on the base rate. Mixing them up has a name too: the confusion of the inverse.
When the base rate changes, everything changes
Suppose the same test is used on people who already have symptoms, and among them the disease affects 1 in 10.
Picture 1,000 of them: about 100 have the disease, and nearly all test positive. Of the 900 who don’t, 1%, about 9, are wrongly flagged. Now about 100 of 109 positives are real: over 90%.
Same test, same accuracy, completely different meaning. That’s why doctors use tests differently for screening the general population and for confirming a suspected diagnosis, and why a positive screening result is usually followed by a second, different test.
Where else it shows up
- Security screening. An alarm that’s 99% accurate at spotting a threat will mostly flag innocent people if threats are very rare.
- Spam filters, fraud alerts, facial recognition: any system hunting for something rare among lots of normal cases.
- Stereotypes. Judging someone’s job from a description, while ignoring how many people do each job, is the same mistake.
The formal version
This reasoning is exactly what Bayes’ theorem does. It combines how likely the evidence is under each possibility with how likely each possibility was to begin with. The crowd-of-people method is the same calculation without the notation, and it’s much harder to get wrong.
Don’t panic, don’t ignore it
None of this means you should ignore a positive result. It means you should ask two questions: how common is this in people like me, and what’s the next test? A doctor will usually be asking the same things.
Test yourself
The free probability test includes this question and nine other classic traps. Our free course on what a positive result on a 99% accurate test really means goes through it step by step.