Statistics test

Q: What does a p-value actually mean?

How surprising your data would be if there were no real effect. A small p-value means the data would be unusual under that assumption. It isn’t the chance the result is wrong.

Q: Is this good practice for a stats exam?

It covers ideas that come up on most intro statistics courses and exams, like AP Statistics and first-year university stats, but it tests understanding rather than calculation.

Q: Why doesn’t a bigger country need a bigger poll?

A random sample’s precision depends on its size, not the size of the population it’s drawn from, the same way a spoonful of well-stirred soup tells you its saltiness whatever the size of the pot.

Q: How many questions are in the statistics quiz, and how long does it take?

Twenty multiple-choice questions, about nine minutes. After each one you see whether you were right and why, and at the end you get a score, a breakdown by topic and the named mistakes that caught you.

Q: What does the margin of error in a poll mean?

How far the poll’s figure could plausibly be from the true one because only a sample was asked. A poll of 1,000 people has a margin of about 3 points on each share, so a lead of 4 points between two parties is within the noise.

Question 1

Towns that sell more ice cream have more drownings. What’s the most likely explanation?

Accepted Answer

Hot weather drives both of them.

Answer

People buy ice cream after a drowning.

Answer

Eating ice cream makes swimming riskier.

Question 2

A study reports p = 0.03. What does that mean?

Accepted Answer

With no real effect, data this extreme appears 3% of the time.

Answer

There’s a 97% chance that the effect found is a real one.

Answer

There’s a 3% chance the finding happened by pure luck.

Question 3

The pupils who scored worst on a test get extra tutoring, and on the next test they score better. What can you conclude?

Accepted Answer

The low scorers were likely to improve anyway.

Answer

The tutoring worked: their scores went up.

Answer

The second test must have been the easier one.

Question 4

In a town, most people earn around $40,000, but a few earn millions. Which is higher?

Accepted Answer

The mean income.

Answer

They’re about the same.

Answer

The median income.

Question 5

In the Second World War, analysts studied bombers that came back full of bullet holes, mostly in the wings and tail. Where should the extra armour go?

Accepted Answer

Where the returning planes weren’t hit.

Answer

On the wings and tail, where the holes are.

Answer

Spread evenly, since hits land everywhere.

Question 6

A random poll of 1,000 people in a country of 3 million has a margin of error of about 3 points. For the same margin in a country of 300 million, you’d need…

Accepted Answer

About the same number of people.

Answer

About 10 times as many people.

Answer

About 100 times as many people.

Question 7

Hospital A has a better survival rate than hospital B for mild cases, and for severe cases. What can you say about their overall survival rates?

Accepted Answer

B’s could still be higher.

Answer

They must be about equal.

Answer

A’s must be higher too.

Question 8

A study gives a 95% confidence interval for the average adult height of 170 to 176 cm. Which reading is right?

Accepted Answer

The method used catches the true average in 95% of samples.

Answer

If you repeated the study, 95% of new averages would land between 170 and 176 cm.

Answer

95% of adults are between 170 and 176 cm tall.

Question 9

A price goes up 50%, then down 50%. Compared with where it started, it’s now…

Accepted Answer

25% lower.

Answer

Back where it started.

Answer

25% higher.

Question 10

A huge study finds a new pill lowers blood pressure by half a point, with p < 0.001. What does that tell you?

Accepted Answer

It’s probably real, but likely too small to matter.

Answer

It’s a large effect, worth taking the pill for.

Answer

It’s important, since the p-value is so small.

Question 11

A poll of 1,000 people has a margin of error of about 3 points. It shows Party A on 52% and Party B on 48%. What can you say?

Accepted Answer

It’s too close to call: the gap could be chance.

Answer

A leads by exactly 4 points in the population.

Answer

A is ahead, since 52 is more than 48.

Question 12

A study tests 20 unrelated foods for a link to cancer, using p < 0.05 as the bar. If none of the foods matters, how many ‘significant’ links should you expect?

Accepted Answer

About one, from luck alone.

Answer

About ten, as each one is a coin flip.

Answer

None: p < 0.05 means 95% certainty.

Question 13

Your exam result says you scored in the 90th percentile. What does that mean?

Accepted Answer

You did better than about 90% of those who sat it.

Answer

You’re 90% sure to pass next time.

Answer

You got 90% of the questions right.

Question 14

Two measurements have a correlation of exactly 0. What can you conclude?

Accepted Answer

No straight-line link, though a curve could exist.

Answer

Changing one can’t have any effect on the other.

Answer

They’re completely unrelated to each other.

Question 15

In a class, the 30 girls average 80 on a test and the 10 boys average 70. What’s the class average?

Accepted Answer

77.5, nearer the girls’ average.

Answer

75, halfway between the two.

Answer

80, because the girls scored highest.

Question 16

A magazine asks its readers to mail back a survey, and 90% of the replies say they love the magazine. What can it say?

Accepted Answer

That 90% of those who replied love it.

Answer

That 90% of its readers love it.

Answer

That about 90% of everyone would love it.

Question 17

A drug cuts the risk of a heart attack by 50%. Without it, 2 in 1,000 people have one over five years. Among 1,000 people who take it, how many heart attacks are prevented?

Accepted Answer

About one: from 2 to 1.

Answer

About 500, half of all the people.

Answer

About 50, five percent of the group.

Question 18

Two datasets have the same mean, the same spread and the same correlation. What does that tell you about how they look when plotted?

Accepted Answer

They can look completely different.

Answer

That one of them must contain an error.

Answer

That the two plots must look very similar.

Question 19

A DNA profile found at a crime scene matches a suspect. One person in 1,000 has it, and the city has a million people. A lawyer says: ‘There’s only a 1-in-1,000 chance he’s innocent.’ What’s the flaw?

Accepted Answer

A thousand residents match, so he’s one of many.

Answer

None: a one-in-a-thousand match is conclusive.

Answer

The odds of innocence are actually 1 in 100.

Question 20

People who take vitamin supplements live longer on average than people who don’t. What’s the best conclusion?

Accepted Answer

Takers may differ in other ways, like diet and exercise.

Answer

Supplements shorten life, since ill people take them.

Answer

Supplements extend life, so everyone should take them.

Statistics test

Towns that sell more ice cream have more drownings. What’s the most likely explanation?

About this test

Questions people ask

Read more

More tests