When Analyzing Statistics, Journalists Often Miss the Mark

Pssst: Want a magic penny? One that can correctly predict 15 out of 17 presidential elections?

Rebecca Goldin knows where to find one.

But her coin isn’t infused with otherworld powers, or driven by divine intervention. Rather, as the George Mason University professor of mathematical sciences said, it’s the product of simple statistical chance: If you flip enough coins enough times, and then winnow out the ones that don’t land the way you want them to, you’ll eventually find one that has the amazing “predictive” qualities you set out to find.

Such are the realities of statistics – and the likelihood that something happened by chance.

In a presentation with National Press Foundation fellows exploring the latest in cancer research, Goldin described the mistakes that are made when people confuse causation and correlation; gave suggestions on how to separate legitimate findings from hype; and offered specific tips for journalists trying to make sense of the blizzard of studies and journal articles that document the latest in medical experiments.

Among them:

  • When reading a new journal article, pay attention to the summary, abstract and the conclusion. Many writers ignore the conclusions. And realize that the abstract will tell you the result but it hardly ever hints at any limitations in the study.
  • Check for significance, by checking if what are known as “p-values” are small enough.
  • Realize that something may be statistically significant but isn’t really clinically significant. That means the findings likely didn’t happen by chance – but they also won’t have much impact on the population of potential patients.
  • Use PubMed to find other literature that’s been published on the topic.
  • Use Stats, a statistical literacy organization for which Goldin is director, to find statisticians willing to help decipher medical and academic literature.


This program is funded by Bayer. NPF is solely responsible for the content.

Subscribe on YouTube
More Presentations
Help Make Good Journalists Better
Donate to the National Press Foundation to help us keep journalists informed on the issues that matter most.