When it Comes to Polls, Verify Them – and Beware the Hype

By Chris Adams

Not all polls are created equal – something reporters itching for another tweet or story need to remember.

“Just because you get a poll, don’t think you need to write about it,” said Emily Swanson, polling editor for The Associated Press. “I tell people not to cover polls more often than I tell them to cover them.”

In a session with Paul Miller fellows, Swanson and Scott Clement, polling director for The Washington Post, laid out the do’s and don’ts to understanding, analyzing and reporting about political polls.

Some of the factors that need to be considered:

__Who is behind the poll? You should be able to figure out when, how and by whom a poll was conducted in a methodology section, Clement and Swanson said. That includes understanding how many people were interviewed; when, where and how they were interviewed; and how they were selected.

The relatively new breed of robo-polls – technically called “interactive voice response,” or automated, polls – use different types of voting lists and different kinds of dialing strategies. The quality of such polls is all over the map.

“Think of it like a thin sheet of ice,” Clement said. “You might be able to tread on it a little bit to get a sense of a race.” But leaning too much on such polls can be risky.

__Who might not be on the call lists? Some people are tied to landlines, while others only have cell phones: 52 percent of American adults don’t have a landline phone at home, and of those 25-34 years old, the figure is 71 percent.

__How were questions asked – and in what order? How were the questions worded? A crafty pollster can produce different results by changing the order or wording of questions, prompting people to be inclined or disinclined to a particular position.

__Was there a strategic reason for releasing the poll? Campaign officials will, of course, be more inclined to release a poll that suggests their candidate is competitive. But what about all the polls that showed that wasn’t the case?

Clement and Swanson talked through other key things to understand, including what the margin of error actually means and when you can say a candidate is leading.

In the 2016 election that Hillary Clinton was widely expected to win, national polls were actually pretty close, predicting her slender popular-vote margin. But state-level polls were far shakier, thus missing Donald Trump’s electoral victory.

More Presentations
Help Make Good Journalists Better
Donate to the National Press Foundation to help us keep journalists informed on the issues that matter most.