Science reporters on deadline have to read quickly, analyze quickly and write quickly. And generally they are good at the reading and writing part.
Less so the analysis – meaning, whether they truly understand and convey the statistical tests that underlie medical and scientific research.
In a session with National Press Foundation fellows, Kevin McConway, an emeritus professor of applied statistics at the Open University in the United Kingdom, gave a tutorial on how to understand one thing that journalists often fear: math.
McConway (bio, Twitter) explained the basic structure of a scientific paper, highlighting the portions – the abstract and discussion – that are most helpful for journalists, and the sections – such as the methods – that are generally of more interest to other scientists.
He also cautioned fellows on the red flags that signal a study might not be worth the big headlines the sponsor of the study would love to see. Among them: language that says something “causes” or “is linked with” something else; an animal study that extrapolates to humans; a study of borderline or no statistical significance; lack of adequate controls; a study that is not peer reviewed; or a study that involves a niche group of subjects and therefore can’t be generalized to a wider audience.
He also told the journalists to recognize the difference between relative and absolute risk. It’s true that something might double the risk of something. But if the baseline is very low then not many more people will be affected.
“The absolute risk is what actually matters,” he said. “Twice of not very much is still not much.”
He also gave an overview on the different kinds of scientific studies – from case studies to randomized controlled studies. And he told them what to look for when assessing the statistical tests – the “p values” that indicates the likelihood that something happened by chance.
He was joined by Fiona Lethbridge (bio, Twitter) of the Science Media Centre in London, a nonprofit organization that connects scientists with journalists on the key scientific issues of the day. The center has a list of 10 best practices for journalists writing about scientific issues.