5 takeaways:
➀ Vet your poll the way you would any source. Emily Swanson, who directs political polling for The Associated Press, said that a poll is like a source: “If you don’t know anything about a source, should you trust it?” she asked. When it comes to reporting on a poll, you should figure out when, how and by whom it was conducted. You should also know how many people were interviewed, when the interviews happened and how subjects were selected. “If you can’t answer these questions, you have no starting point to judge a poll’s quality or newsworthiness,” she said. If pollsters are hesitant to release the details you need, remind them of the industry’s disclosure standards, as set by the American Association for Public Opinion Research.
➁ The decline of the landline has dramatically changed how pollsters interact with subjects. Some 65% of American adults don’t have a landline phone at home and more than 80% of people aged 25-34 only have a cellphone. (Not one of the 20 NPF Paul Miller fellows in the session, had a home landline.) Among those over 65, 37% only have a cellphone. Pollsters can reach cellphone users through random digit dialing or voter lists, as well as online polls that are based on probability sampling or on subjects’ willingness to opt-in to the survey. But be warned: Research has found that for some issues, people give very different responses in online polls compared with phone ones.
➂ Knowing which candidate is “leading” in a poll requires understanding the margin of error. The AP’s standards on when to write that a candidate is “leading” or when a race is essentially a tie are based on the margin of error. Swanson said. If the difference between two response options is more than twice the margin of error, then the poll shows that one candidate is “leading.” If the difference is at least equal to the margin of error but no more than twice the margin of error, then one candidate can be said to be “apparently leading” or “slightly ahead.” If the difference is less than the margin of error, the race is close or about even. Also: Don’t use the term “statistical dead heat,” Swanson said, as it’s not an accurate reflection of what the numbers mean.
➃ There are more than just horses in the race. Reporters are understandably fascinated with horse-race coverage of campaigns, but polling is equally important in deciphering the mood of the country through issues polling. Even so, asking people their views on hot-button issues can be tricky – particularly in how they are asked. For example, different polls aimed at gauging support for the Trump administration’s travel ban in 2017 yielded very different results depending on the wording and survey method used. Look at polls that track attitudes about issues over time. Natalie Jackson, director of research for the Public Religion Research Institute, said one of the best ongoing polls on social issues is the General Social Survey, from the University of Chicago, which has been measuring Americans’ attitudes since 1972.
➄ It’s not just what people were asked, but when – and in what order. Swanson said that question order can have a big influence on the answers people give pollsters. A straightforward poll sponsored by the American Petroleum Institute asked: “Do you support or oppose offshore drilling for domestic oil and natural gas resources?” That question, however, came after respondents were asked to indicate whether they agreed or disagreed with several statements that framed the issue in a pro-drilling way, including “Increased production of domestic oil and natural gas could lead to more jobs in the U.S.,” and “Producing more domestic oil and natural gas could help lower energy costs for consumers.,”
Speakers:
Natalie Jackson, Director of Research, Public Religion Research Institute
Emily Swanson, Director of Public Opinion Research, The Associated Press
This program, part of the Paul Miller Fellowship, was funded by the Walter and Karla Goldschmidt Foundation, Johnson & Johnson and its employees, in honor of the late Wanda Moebius, and other donations to the National Press Foundation, which is solely responsible for its content.
