Election polls have come under fire the last few campaign cycles, yet despite their limitations, many audiences still express interest in them. If you are going to cover political polling, here’s what to know according to Jim Henson, director of the Texas Politics Project and professor in the University of Texas, Austin, department of government. (Transcript | Video)
5 takeaways:
➀ Pollsters should be transparent with journalists and the public. Everyone knows to look at sample size, but journalists must also dig into who’s behind a poll. “What is the source? Who are the producers? Who are the sponsors of the data that you’re being asked to use? … Your readers need to know where this came from, and they need to know a little bit about the organization,” Henson said. Journalists should ask for the actual data sets of the poll in order to verify the results, as well as the methodology and question-wording.
➁ Explain to audiences that how you word a question matters. “If we say, ‘X amount of people want gun laws made more strict, Y want them made less strict. Z want them left as they are,’ you need to see exactly how that question was worded,” Henson said. For instance, most strict and less strict should be randomly switched. “It’s easy to encourage the expression of attitudes that don’t really exist or are very shallow and are easily moved by the way that you asked the question,” he said. “During the Trump presidency, all you have to do is put the president in a question linked to some policy question at the state level, and you’re just moving the results.”
➂ Calling a poll a “snapshot in time” is a cliché because it’s true, Henson said, and a reminder to avoid over-interpretation. “During horse-race polling and during election polling, it’s very easy to over-interpret,” Henson warned. “We do things with 95% confidence intervals. That means theoretically one in 20 things could just be an outlier … that’s a pretty high probability that something could be off.” Look at trends and replication. “If you’re talking to a pollster, you should say, ‘Hey, has anybody else polled on this? Are there any data in and around this that are similar to this result that we’re talking about that you guys just released?’ Now, some people aren’t going to want to tell you. But if they don’t tell you and then you find it, you know something about your source.”
➃ Policy polling can offer insights into voter attitudes. Understanding voters’ feelings about gun rights, LGBTQ rights or marijuana legalization can lead to stories about whether lawmakers are ahead or behind the public. “If public opinion is really moving … then how come it’s getting no traction in the legislature?” Henson asked. “It requires a certain amount of familiarity with the dynamics to the system, where the choke points are, who the gatekeepers are, to understand where those bills are getting killed.” “Everybody’s dislike of horse race coverage is that people get so focused on [it] that they really miss how valuable sustained polling on policy is,” Henson said. Pollsters started asking questions about gay marriage in 2008 and were able to see shifts over time.
➄ What is a ‘likely voter’? “Know the difference between the concentric circles of the voting-age population, the eligible electorate, the registered electorate, the electorate on election day that actually shows up,” Henson said. “Somewhere cutting across there and congealing during the period of early voting and on election day is the actual electorate, but we don’t know what that is.” Understanding who finally casts a vote is particularly daunting when there are 51 different election laws. Whether one deems it voter suppression or election integrity, “in a more technical sense, states are creating filters on the electorate,” Henson said.
The Statehouse Reporting Fellowship was sponsored by Arnold Ventures. NPF is solely responsible for the content.
