The Baloney Detection Kit

Carl Sagan popularized a pocket toolkit for claims that want your trust too quickly. The spirit of it is simple: extraordinary claims need evidence that is as strong as the claim is strange, and that evidence has to be able to fail.

A working kit asks a few questions every time. Can someone else check this? What else could explain it? Would I still want this to be true if the test went the other way? Is every link in the chain there, or am I skipping from a story to a conclusion?

Authority, sincerity and a vivid testimonial are not nothing. They are also not a test. “It cured me overnight” is a story. A comparison of people who used the remedy with people who did not is the beginning of a test.

The overnight crystal

A friend says a crystal ended their flu by morning. They are not lying about how they felt. Flu often turns a corner overnight anyway. One recovery, with no comparison and no way to be wrong, cannot carry a claim that crystals cure influenza.

The kit does not sneer at the friend. It asks what we would expect if the crystal did nothing.

Try this today Next time you meet a dramatic claim, ask what a failed test would look like and whether anyone ran it.