# Survivorship Bias: The Planes That Didn’t Come Back Thinking · 8 min read · updated 2026-10-29 > Abraham Wald saw what the military missed: the planes that never came back. How the same blind spot warps success stories, investing and old buildings. ## The short answer Survivorship bias is judging a group by the members that made it through, because the ones that didn’t have dropped out of sight. The returning bomber had holes where a plane could be hit and still fly home; the planes hit in the engines never came back to be counted. To correct for it, ask what had to be true for something to be in your data, and who’s missing. In 1943, the American military had a problem it could measure. Bombers were flying out over Europe and too many weren’t coming back. Armour plating would help, but armour is heavy, and a heavier plane flies shorter and slower and carries fewer bombs. So the question was where to put the weight. The Air Force had data. Engineers inspected the planes that made it home and mapped every bullet hole. The pattern was clear: holes clustered across the wings, the tail and the main body of the fuselage. The engines and the cockpit were hit far less. The obvious plan was to bolt armour to the parts that were being shot up. Then someone took the data to a statistician named Abraham Wald, who worked in a classified group at Columbia University, and Wald said something that sounds almost too simple. Put the armour where the holes *aren’t*. His reasoning was this. These were the planes that came back. A plane with a hole in its wing could still fly home. A plane with a hole in its engine often couldn’t. The engines looked safe only because the planes that were hit there were lying in fields in Germany, absent from the sample. The clean engines weren’t good luck. They were a body count. ## How the same hits give different holes Suppose you could watch every hit, not only the holes on the survivors. You’d see bullets landing roughly everywhere, and the engines taking as many as the wings. The pattern flips only when you add the filter: you get to see a plane only if it survived. *(An interactive illustration belongs here. Try it on the page: https://cognosc.com/blog/survivorship-bias)* That’s the whole mechanism. Survivorship bias is what you get when the thing you’re studying has been run through a filter that removes the failures, and you forget the filter is there. The data isn’t wrong. Every hole on those returning planes was really there. The error is in the question you think the data answers. The trick for spotting it is one question, and you can ask it of almost anything: **what had to be true for this to be in front of me?** A plane had to survive. A company had to stay in business. A book had to find a publisher. A building had to stay standing. Whatever the filter was, the things you can see are the ones that got through it. **The one question** What had to be true for this to be in my sample, and who isn’t here because they didn’t make it? ## Where it’s hiding in everyday life ### Success advice Take any shelf of business books. They study winners, find what the winners had in common, and offer it as a recipe: they took big risks, they ignored the doubters, they dropped out, they got up at five. Now ask the Wald question. How many people took big risks, ignored the doubters, and lost everything? How many people got up at five and still went bust? They’re not in the book, because nobody writes a case study about a business that quietly closed. Yet they’re far more numerous than the winners. If a thousand people take the same long shot and one succeeds, that one will give talks about courage. The strategy might be sound, or it might be a lottery ticket. You can’t tell from the survivor alone. *(An interactive illustration belongs here. Try it on the page: https://cognosc.com/blog/survivorship-bias)* None of this means winners learned nothing. It means a pattern found only among winners tells you what they did, not what made the difference. To learn from them, you need the losers who did the same things, so you can compare. ### Investing Fund managers love to report the average performance of their funds. The catch is that poor funds tend to be closed or merged into better ones, and when they go, they usually vanish from the databases too. What’s left is a list of survivors. In a well-known study of US mutual funds, Elton, Gruber and Blake estimated that survivorship bias overstated returns by roughly a percentage point a year. That sounds small until you compound it over decades. It’s the difference between a retirement pot that looks comfortable on paper and one that’s smaller in reality. *(An interactive illustration belongs here. Try it on the page: https://cognosc.com/blog/survivorship-bias)* The same filter hides in *“this fund beat the market ten years running”*. Start a thousand funds, let luck do its work, and some will have a ten-year streak. Close the rest and you can print their record in an advert. ### “They don’t make them like they used to” Old buildings look sturdier than new ones, old furniture looks better made, old cars look more charming and more solid. There’s something to that, because good craftsmanship does last. But look at what you’re comparing: a few centuries’ worth of the best-built survivors against everything built last year, good and bad alike. The jerry-built terraces and rickety chairs of 1850 fell apart long ago and have been quietly removed from the sample. The past looks like it was built better partly because only the well-built parts of it are left. ### “What doesn’t kill you makes you stronger” A harsh upbringing, a brutal training regime, a crash diet. Some people who went through it are tougher and say it made them so. But if a hardship removes the people who couldn’t take it, then everyone you meet afterwards looks as though they thrived. The same logic hides in “my grandfather smoked and lived to 95”. Of course you’ve heard from the one who lived to 95. The ones who died at 60 aren’t around to tell you the story. ### Science and the news Scientific journals publish studies that found something. A study that tried a promising treatment and found it did nothing is far less likely to be written up, let alone printed, so it ends up in a drawer. The published literature is a survivor list. If twenty teams test the same idea and one gets an exciting result by luck, that is the one you’ll read about. This is called the file-drawer problem, and it is one reason a surprising finding should be treated as a lead, not a fact, until someone else gets the same result. The news does it too. Planes landing safely aren’t a story; one that crashes is. That cuts the other way, and gives a different bias, because rare dramatic events get so much attention that they feel more common than they are. Both come from the same place: what reaches you has been chosen, and not by chance. ### Reviews, rankings and “top 10” lists Restaurants that closed aren’t on the review site. Apps that failed aren’t in the store. Hedge funds that blew up aren’t in the ranking. Any list of what currently exists has been filtered by what survived long enough to be listed, and that makes the average of the list look better than the average of what was ever tried. ## When it’s the opposite of what you’d expect The most interesting thing about Wald’s insight is that it ran against the evidence everyone could see. When data tells you where the damage is, the instinct is to protect the damaged parts. His point was that, in this sample, **absence was the signal**. The places with no holes were the places where a hit was fatal. That flips a common intuition: that more evidence means more certainty. If the evidence is filtered, more of it can make you *more* wrong, because you become surer of a picture that’s missing a piece. Engineers use this idea whenever they study crashes, not just flights that landed safely. It’s the reason aviation investigates every accident, and why a field with a culture of examining its failures tends to improve faster than one that celebrates its successes. ## How to catch it before it catches you You can’t fix a filter you haven’t noticed, so most of the work is noticing. A few habits help: - **Ask who’s missing.** For any group you’re studying, ask what process decided who ended up in it. Who dropped out, closed, died, was deleted or never got published? - **Find the denominator.** “Seventeen of twenty billionaires” means little. “Seventeen of every thousand people who tried” would mean a lot. Whenever you see a count of winners, look for the count of everyone who tried. - **Go looking for the graveyard.** Failed startups, closed funds, studies that found nothing and never got published. They exist, even if they’re harder to find. - **Distrust “always” and “all of”.** *All successful people do X* is almost always a sentence about the survivors. - **Compare with the people who did the same thing and failed.** That’s the only way to see whether the strategy helped or whether it was just common. ## A harder case: the school that “produces” success Imagine a school whose alumni include a prime minister, three Olympians and a Nobel winner. Does the school make people successful? Maybe. But the school also chooses who it admits. If it takes only children who were already bright, ambitious and well-connected, those children would likely have done well anywhere, and the school’s record is partly just a record of whom it picked. Add that you hear about the famous alumni and not the ones who drifted into jobs nobody writes about, and the school’s halo grows brighter still. To work out what the school adds, you’d need to compare its pupils with similar pupils who went elsewhere. That’s harder than counting famous names, which is exactly why people count famous names. ## What it doesn’t mean It doesn’t mean success stories are worthless or that you can’t learn from people who made it. It means you should learn from them *carefully*: take their stories as hypotheses worth testing, not as proof. It also doesn’t mean everything is a lottery. Some things really do separate winners from losers. Survivorship bias just stops you from knowing which, until you look at both sides. When someone offers you a pattern, the useful response isn’t cynicism, it’s a question: *compared with what, and who’s missing?* And there’s a quiet upside. Once you’ve seen it, you notice it everywhere: in the ranking that only shows today’s leaders, in the “classic albums” of a decade that only kept the good ones, in the older relative whose habits “never did them any harm”. It makes the world slightly less impressive and a lot more intelligible, which is a fair trade. ## What to remember - The data you can see has already been filtered by survival, so what you see is biased even when every number is true. - Ask “what had to be true to be in this sample?” and “who dropped out?” before you copy anyone. - Success advice, fund averages and “they don’t make them like they used to” all hide a graveyard. - Look for the failures that did the same thing, or the denominator, before trusting a pattern in winners. - Sometimes the best evidence is in what’s absent: the holes that aren’t there. ## Questions people ask ### What is survivorship bias in simple terms? It’s a mistake of looking only at the people or things that made it through some process and ignoring those that didn’t. Because the failures have vanished from view, the survivors look more typical and more skilful than they are. ### Did the Abraham Wald bomber story really happen? The core is real: Wald and the Statistical Research Group worked out how to estimate aircraft vulnerability from the damage on planes that returned, and their memos survive. The tidy tale of one flash of insight has been polished in the retelling, and the real work was heavy mathematics on damage data. ### Is survivorship bias the same as selection bias? It’s a type of it. Selection bias is any way the sample you’re looking at differs from the group you care about. Survivorship bias is the version where the selection is done by survival: the sample is whoever lasted. ### How does survivorship bias affect investing? Fund databases often drop funds that closed or merged, usually the poor performers, so the average of what remains looks better than what investors actually got. One well-known study found the effect was worth roughly a percentage point a year, which compounds into a large difference. ### How do you avoid survivorship bias? Look for the full denominator: everyone who started, not just everyone who finished. Ask what filter the data passed through and who didn’t get through it. If the failures are hard to find, treat the pattern as a hint, not a rule. ## Sources and further reading - Wald, A. (1943). A Method of Estimating Plane Vulnerability Based on Damage of Survivors. Statistical Research Group, Columbia University (reprinted 1980 by the Center for Naval Analyses). - Mangel, M. and Samaniego, F. J. (1984). Abraham Wald’s work on aircraft survivability. Journal of the American Statistical Association, 79(386), 259–267. - Wallis, W. A. (1980). The Statistical Research Group, 1942–1945. Journal of the American Statistical Association, 75(370), 320–330. - Elton, E. J., Gruber, M. J. and Blake, C. R. (1996). Survivorship bias and mutual fund performance. Review of Financial Studies, 9(4), 1097–1120. - Ellenberg, J. (2014). How Not to Be Wrong: The Power of Mathematical Thinking. Penguin. --- Canonical page: https://cognosc.com/blog/survivorship-bias From Cognosc (https://cognosc.com): free tests, articles and an AI tutor that has you reason things through.