How to Interpret Scientific Studies

Science reporters on deadline have to read quickly, analyze quickly and write quickly. And generally they are good at the reading and writing part.

Less so the analysis – meaning, whether they truly understand and convey the statistical tests that underlie medical and scientific research.

In a session with National Press Foundation fellows, Kevin McConway, an emeritus professor of applied statistics at the Open University in the United Kingdom, gave a tutorial on how to understand one thing that journalists often fear: math.

McConway (bio, Twitter) explained the basic structure of a scientific paper, highlighting the portions – the abstract and discussion – that are most helpful for journalists, and the sections – such as the methods – that are generally of more interest to other scientists.

He also cautioned fellows on the red flags that signal a study might not be worth the big headlines the sponsor of the study would love to see. Among them: language that says something “causes” or “is linked with” something else; an animal study that extrapolates to humans; a study of borderline or no statistical significance; lack of adequate controls; a study that is not peer reviewed; or a study that involves a niche group of subjects and therefore can’t be generalized to a wider audience.

He also told the journalists to recognize the difference between relative and absolute risk. It’s true that something might double the risk of something. But if the baseline is very low then not many more people will be affected.

“The absolute risk is what actually matters,” he said. “Twice of not very much is still not much.”

He also gave an overview on the different kinds of scientific studies – from case studies to randomized controlled studies. And he told them what to look for when assessing the statistical tests – the “p values” that indicates the likelihood that something happened by chance.

He was joined by Fiona Lethbridge (bio, Twitter) of the Science Media Centre in London, a nonprofit organization that connects scientists with journalists on the key scientific issues of the day. The center has a list of 10 best practices for journalists writing about scientific issues.

This program is funded by Fondation Ipsen and under the aegis of the Fondation de France. NPF is solely responsible for the content.

Kevin McConway
Emeritus professor of applied statistics, School of Mathematics and Statistics, The Open University
Fiona Lethbridge
Senior press officer, Science Media Centre
More Presentations
Help Make Good Journalists Better
Donate to the National Press Foundation to help us keep journalists informed on the issues that matter most.
You might also like
Addressing Health Inequalities
Confidence in Scientists
Developing Vaccines
How to Write About Vaccine Deniers
New Thinking on Pain
Scientific Spin
Single Cell Technology
Studying Hate
The Health Impact of Torture
The New Neurobiology of Addiction
The Science of Science Communication
Understanding Global Health Inequalities
When Science Is Bad
WHO’s Emergency Response System