PA Senate Polling: Burn After Reading?
April 9, 2010 · 9:00 AM EDT
With the proliferation of polling this cycle, political junkies, journalists and party strategists are swimming in a sea of numbers. With each new survey that is released, the party that is ahead touts the results while the party that trails disparages and discounts the pollster. Rinse and repeat.
But divergent poll results aren’t necessarily the result of a partisan agenda. An examination of seven polls over six weeks in the Pennsylvania Senate race demonstrates varying methodologies and, more importantly, a difference of opinion on the makeup of the electorate in November.
Instead of just reading the top-line results, it’s necessary to examine dates and sample sizes and digest the poll in context. Unfortunately, context is often a casualty in the current 24/7 churn of the news cycle.
Beginning in late February, a Quinnipiac University poll showed Sen. Arlen Specter (D) ahead of former Rep. Pat Toomey (R) by 7 points in a general election matchup. A week later, Susquehanna Polling and Research (R) showed Toomey with a 6-point advantage. A couple days after that, a Research 2000 survey done for the liberal Web site Daily Kos gave Specter a 6-point edge. Five days later, Rasmussen Reports had Toomey up by 9 points, and then Franklin and Marshall College showed the Republican with a narrower 4-point lead.
More recently, a Public Policy Polling (D) survey showed Toomey with a statistically insignificant lead and, in a Quinnipiac poll released Thursday, the former GOP Congressman was up by 5 points. Click here to see the chart of all seven polls and details of the samples.
So what happened? Why so much movement in a race seven months away from Election Day?
Even though the polls were taken against the backdrop of the escalating health care fight on Capitol Hill, the Senate campaigns were not engaged in heavy paid media campaigns that often affect poll results.
Instead, the varying results were likely because of a difference in methodologies, and if the margins of error are applied correctly, the polls are not that far off from each other.
At this stage of the cycle, pollsters are still looking at different samples of people. Susquehanna, Research 2000, and Rasmussen surveyed likely voters while Quinnipiac, PPP, and Franklin and Marshall polled registered voters. Comparing the two different groups is wrong.
“I don’t know why people pay attention to polls of people who aren’t going to vote,” said Jon Lerner of Red Sea Communications, who is polling for Toomey.
“Registered samples, as you get closer, are useless,” according to Research 2000’s Del Ali, who added that they aren’t necessarily a problem months out from Election Day.
“The importance of likely voters gets greater as Election Day approaches. [But] it seems a stretch that [the likely voter pool] you’re envisioning today will hold up to Election Day,” explained Berwood Yost, head methodologist of the Franklin and Marshall College poll. “Voters aren’t sure about the campaigns and aren’t paying attention.”
Franklin and Marshall and Quinnipiac will be switching to likely voter samples later this cycle.
So, how does this affect the survey results?
“I believe that the likely voter model will be more favorable to Republicans,” Scott Rasmussen told Roll Call, noting the increased enthusiasm of the GOP and general historical midterm trends against the party in power. “I don’t believe that there is a Republican resurgence,” added Rasmussen, whose polls are often criticized as being GOP-friendly. “It’s a resistance to Democratic rule … the inverse of 2006 and 2008.”
But that doesn’t explain Research 2000’s poll of likely voters, which had Specter with a significant lead. The age of the electorate is becoming a critical point of contention.
“That’s the magic question,” laughed Ali of Research 2000, whose March 8-10 survey had 18-to-29-year-olds making up 14 percent of the sample and Specter leading by 6 points, 47 percent to 41 percent. Younger voters (18-29) made up 18 percent of the electorate in 2008 and 11 percent in the 2006 midterm elections, according to exit polling.
“I don’t look at exit polls as gospel,” Ali said. “I’m not so much etched in stone that every midterm electorate will be the same.” The pollster has seen the number of younger voters decrease over the past year in other polls he’s conducted. Younger voters made up 16 percent of the sample in Ali’s August poll in Pennsylvania.
Republicans believe the midterm electorate will be significantly older than the one that showed up to vote in 2008, and consequently more friendly to GOP candidates. Steven Shepard of National Journal recently compared R2K’s sample with Susquehanna, which had 18-to-29-year-olds making up just 5 percent of the sample and Toomey with a 42 percent to 36 percent advantage.
Rasmussen evaluates exit polls and adjusts the sample compared to trends in his monthly national tracking survey, among other factors. Younger voters made up 9 percent of the Rasmussen poll, which had Toomey ahead 49 percent to 40 percent.
However, age does not always determine the outcome of the ballot tests. The February Quinnipiac and March Franklin and Marshall surveys pegged younger voters similarly making up 12 percent and 11 percent of the vote, both sampled registered voters, and each poll had a different leader. The Franklin and Marshall poll was taken at the height of the health care debate and could have picked up GOP enthusiasm.
In some cases, divergent poll results can be explained by the partisanship of the sample. In the Massachusetts special election, partisan pollsters disagreed about who would show up to vote. But the partisan makeup of the sample may not matter as much in the Keystone State.
“The high level of two-party registration masks the divisions within the party,” Lerner said. Party registration usually reflects base support, but the two-party registration is so high that there are many “soft” partisan voters in those numbers.
Pennsylvania has an extremely high rate of two-party registration: 51 percent Democratic, 37 percent Republican and 12 percent others. In comparison, even though Massachusetts is a Democratic state, it’s only 37 percent Democratic, 12 percent Republican and 51 percent unenrolled or others. Identifying the number of independents is critical because those voters helped vault Republicans to victory in the New Jersey and Virginia gubernatorial elections in 2009 and in Massachusetts.
There are other considerations when evaluating polling such as random digit dial versus polling off a voter file, question wording and interactive voice response versus live callers.
Even though party operatives handling polling results like hand grenades, the pollsters themselves are much more deferential.
“It troubles me from an industry standpoint that when polls come out, people instantly decide who to root for,” Rasmussen said. “I believe you learn from all the polls out there.”
By the end of the month, Rasmussen will have polled in every Senate and gubernatorial race in the country and plans to poll at least once a month in the competitive races through Election Day.
“He’s accurate based on his weighting. I feel good about our numbers based on our weighting,” Ali said about Rasmussen. Like any election, this year’s results will boil down to who shows up to vote. “The jury is still out,” Ali added about the November electorate, “How does anyone know that?”