Courtesy of Pack Poll
Emeline McCaleb, a third-year studying political science, included the above graph in her article that portrayed the number of mentions each survey website received on social media in a particular week.
Pack Poll recently began a process of categorizing public polls as either reputable or disreputable based on the methodology of how the poll was conducted.
Pack Poll is a source of NC State polling data that collects information through representative surveys. Mike Cobb, assistant professor of political science, is the faculty adviser for Pack Poll and discussed the motivation for the categorization process.
“There’s a lot of misinformation out there so there’s misleading stories,” Cobb said. “A lot of polls are about public opinion on a variety of issues, so accuracy can’t be the only reason we judge polls as good or bad. We want to know who’s actually doing good, socially sound polling.”
Emeline McCaleb, a third-year studying political science, began contributing to Pack Poll last fall. She wrote an article posted on the Pack Poll website in March reflecting on the current condition of the polling atmosphere.
“We in Pack Poll have been trying to focus on how social media affects polling and just in searching polls on Twitter, a lot of the widely circulated polls are not legitimate,” McCaleb said. “It’s kind of hard to ignore when the most popular poll being shared is one that is from a right-wing news site that doesn’t have any scientific basis at all.”
According to McCaleb, the way that people interact with polls on social media contributes to the development of disreputable polls.
“Obviously there’s a lot of fake news going around, and a lot of people can recognize fake news when it’s obviously factually incorrect,” McCaleb said. “A lot of people don’t, and I don’t blame them, [look] at the methodology and how big the sample size is and what exactly the questions asked for and what order the questions were in because that’s a lot to ask from someone that has more than a passing interest in it.”
Pack Poll intends to create a new awareness of what makes a poll effective.
“I think we’re hoping to kind of call out some of the ones like Drudge, even ones that are seen as more reputable like Rasmussen which President Trump cites a lot as reputable but isn’t,” McCaleb said. “[We plan to] help people see that just because something appears to be real and appears to be representative because 75,000 people responded to it isn’t necessarily representative.”
According to McCaleb, there are simple ways to detect the reputability of a poll without thorough, time-consuming research and she provided some suggestions for how students can do this.
“Making sure that [polls] have methodology available,” McCaleb said. “You don’t have to go in and look, but if they have a link that says this is how we conducted it you can usually trust it. If the only context you see it being used is tweeted out by people that have ‘MAGA’ in their username or just ‘resist’ in their username then it’s probably not trustworthy.”
McCaleb says that even the sample size of a poll can be a telling factor of how reputable it is.
“Sample size around 1,000 is usually pretty legitimate,” McCaleb said. “A huge sample size or a tiny sample size are red flags, I would say.”
Cobb explained the importance of transparency in polling.
“They’re pushing for pollsters to disclose a lot of information,” Cobb said. “Those that don’t [disclose a lot of information] tend to have less accurate polls.”
Students can access McCaleb’s article on the Pack Poll website.