Liberal Blog: We Poll, You Decide
January 28, 2011 · 9:30 AM EST
If you weren’t buried in the avalanche of polling last cycle, beware: More numbers are on the way.
Daily Kos, one of the country’s best-known liberal blogs, recently started its weekly State of the Nation poll measuring President Barack Obama’s job approval rating and other national atmospherics. But the site’s founder is going one step further by releasing the raw data along with the standard top-line numbers.
“Everyone should release the raw data,” Daily Kos publisher and founder Markos Moulitsas told Roll Call in an e-mail. “It would help keep pollsters honest ... but also gives the public the opportunity to slice and dice the information however they want to.”
It’s a lesson that Moulitsas admitted “I learned the hard way,” after he learned his pollster, Research 2000, was releasing questionable figures.
“Democratizing access to our poll results means that everyone gets access to it,” Moulitsas said. “If conservatives want to accuse me of dishonesty or cherry picking of results, well then, they have the ability to dig into the raw data in order to try and prove those accusations.”
Pollsters say Moulitsas could just be compensating for the Research 2000 problems, and it remains to be seen whether he’s starting a trend and whether the raw data are useful to the average political junkie.
“It’s a little bit of chest-thumping,” GOP pollster Adam Geller of National Research Inc. said about the release of the additional numbers. “The fact is that that’s fine and good, but the raw data doesn’t demonstrate anything but a poll was conducted.”
In this case, that’s not an insignificant detail.
Last year, after paying the Maryland-based Research 2000 for dozens of polls, Moulitsas became suspicious that the firm was producing numbers without conducting surveys. After Research 2000 declined to release the raw data to Moulitsas, the Daily Kos publisher severed his relationship with the firm. He’s now suing the pollster in federal court for fraud and breach of contract.
When looking for a new pollster, Moulitsas required that the firm be willing to release the raw data. North Carolina-based Public Policy Polling was a natural fit because of the considerable amount of information that the Democratic-leaning firm normally releases.
“I think the PPP cross-tabs are a useful service because a lot of the public polls tend to only report the top lines or make the subgroups hard to find,” said Democratic pollster Fred Yang of the Garin-Hart-Yang Research Group.
For the Daily Kos State of the Nation surveys, PPP is releasing cross-tabs as well as the answers to all 20 questions from the poll. But to the untrained eye, the data are a thousand rows (one for each respondent) of 20 columns filled with numbers one through four. Even to experienced pollsters, the data fall short of being immediately useful.
“It is really a matter of time. Do we have the staff time to take it and crunch it and see if there are any flaws?” said Democratic pollster John Anzalone of Anzalone Liszt Research, which polls in dozens of House races every cycle.
Multiple pollsters interviewed for this article on both sides of the aisle agreed that the spreadsheet is a long way from being useful and had concerns about the methodology, including PPP’s use of interactive voice response technology, which uses an automated voice instead of a live caller to speak with respondents.
But concerns aside, polls are not cheap, so why wouldn’t campaigns jump at the chance to get free numbers, particularly when Daily Kos/PPP conducts polls in individual races where partisan pollsters have clients?
“The ‘raw’ data is only useful if you are willing and able to take the time to convert it into something. It would take a lot of work to convert this into something that most people could look at,” another GOP pollster said, agreeing with Anzalone. “It never provides the sort of information that would help to make decisions about how to allocate resources or impact our message.”
When making critical campaign decisions, pollsters always rely on their own numbers.
“At the end of the day, what we do is less about the big numbers, little numbers and vote tallies than it is about messaging and strategic positioning,” Anzalone said.
There is a hope among partisan pollsters that the latest effort by Daily Kos/PPP will force other nonpartisan pollsters to release more information. Media and university polls frustrate campaigns because they can drive the media coverage of a race with little or no consequences for being inaccurate.
“If it weren’t for the impression it left on the media and potential donors, we would ignore this stuff completely,” one GOP pollster said about noncampaign polls. But with the overall cost of polling dropping dramatically, media polls aren’t going anywhere anytime soon.
“In the end, I applaud them for releasing the data, and maybe it starts a trend,” Anzalone said.
“I wish it were the beginning of a trend, but I’m not sure how much confidence I have about that,” said Nate Silver, author of the number-crunching FiveThirtyEight blog at the New York Times. “The more detail you release, the more things people may find to criticize: For instance, they may be able to reverse-engineer your likely voter model or your weighting scheme, and pollsters tend to be protective about that stuff.”
Silver was on the front lines of questioning Strategic Vision, which consistently released polling numbers in key races, usually in the South. The polls helped shape the media narrative, but it was unclear who was paying for the surveys. After Silver’s analysis questioning whether polling calls were ever made, the firm stopped releasing numbers.
About the Daily Kos/PPP release, Silver added, “I agree that very few people would actually use the data — but it does contribute to transparency and credibility.”