Lesson · 4 min Steady
How Big a Sample?
In one line Small samples produce striking results by luck, so ask how many before you believe it.
If you flip a coin four times, getting three heads is not unusual. If you flip it four hundred times, getting three hundred heads would be astonishing. Small samples swing widely by luck, and large ones settle near the true rate.
That is why a report that “four of five users loved it” tells little, and why one striking week of sales can be noise. A big effect in a tiny group may vanish when more people are counted.
So ask how many. A large sample does not guarantee the result is right, since it can be biased, but a small one makes luck a serious rival. The smaller the group, the more cautious you should be about a dramatic claim.
The hot streak
A coffee shop owner tries a new blend for a week. Five of six regulars say they love it, and he prepares to order a year’s supply.
His daughter asks how many customers come in a month. About nine hundred. Six is under one percent, and those six were regulars. He orders a small batch and asks all customers to rate it for a month before deciding.
Try this today Next time you read a surprising result, find the number of people behind it before you share it.
Cite or share this lesson
A plain reference you can paste into notes or an essay:
Quest2Reason. “How Big a Sample?.” https://quest2reason.com/learn/lesson/how-big-a-sample. Accessed 13 November 2026.
Sources
- Tversky and Kahneman, “Belief in the Law of Small Numbers” (1971) — Why people expect small samples to look like the whole, and why they do not.
- Wikipedia, “Law of large numbers” — Why large samples settle near the true rate while small ones swing.
- Wikipedia, “Hasty generalization” — Concluding from too few cases.