What “Significant” Means

In everyday speech, “significant” means important. In statistics, it means something narrower: the result would be unlikely if nothing were going on but chance. It says little about how big or useful the effect is.

With a large enough sample, a tiny difference can be statistically significant. A diet that lowers weight by a few grams may pass the test and still be useless. With a small sample, a real and large effect may fail the test because there is too little data.

So ask two questions. Is the result likely to be more than chance? And how big is the effect, compared with what would matter in real life? The first is about confidence, the second is about size.

The small difference

A news story says a new diet plan gives a “significant” weight loss. Sol looks at the study: 20,000 people, and the average difference was 90 grams over six months.

The result was unlikely to be chance, because the sample was huge. But 90 grams is less than a bag of sugar. Sol decides the diet is not worth the cost, even though it is “significant.”

Try this today When you read “significant,” look for the size of the effect and ask whether it matters.