Lesson Learned? The State of Statewide Polling
March 25, 2020 · 8:15 AM EDT
The 2016 Democratic primary in Michigan was a wakeup call. With every recent survey of the state showing Hillary Clinton ahead by double digits, many deemed a Bernie Sanders victory out of the question — until results came in. The Vermont senator’s 23 percentage point overperformance is now considered one of the biggest polling errors in primary history.
There are many causes behind the record-breaking blunder, but the night’s shortcomings can be attributed partially to a drought of contemporary data and the ensuing uncertainty. Only three surveys of Michigan were released in the week preceding the 2016 primary; four years later, pollsters learned their lesson and conducted twice as many polls beforehand, taking some of the suspense out of Joe Biden’s win earlier in March.
It should come as no surprise that having more polls improves predictions. The relationship is key to choosing where surveys are fielded, because even if it would be nice to have 50 polls of all 50 states, pollsters are understandably interested in some places over others. And while such decision-making is consequential for projecting the outcomes of presidential primaries, it also might have an outsized impact on forecasting the general election.
Based on the final Inside Elections race ratings of the 2016 Electoral College, a state’s “battleground” status largely determined how often it was polled. The five states with the most surveys in the run-up to that November — Florida, North Carolina, Pennsylvania, Ohio, and Virginia — comprised one Toss-Up, two Tilting Democratic, one Leaning Democratic, and one Favored Democratic.
The broad picture broke down uniformly:
- Ohio, the sole pure Toss-Up, had 35 separate polls over the course of the election
- States Tilting Democratic/Republican averaged 32 polls
- States Leaning Democratic/Republican averaged 26 polls
- States Favored Democratic/Republican averaged 15 polls, and
- States Safe Democratic/Republican averaged only 5 polls
There in total counted 623 state-level surveys (310 of which were conducted the month preceding the race) in the RealClearPolitics archives, between two to three times the number of national polls.
Yet, due to diminishing returns associated with consistently studying the same area, some states became over- or under-polled. Though Trump won North Carolina by 3.7 percentage points, pollsters surveyed there more than any other except Florida. And during the final weeks of the race, 12 separate polls of Utah gauged interest in potential spoiler candidate Evan McMullin.
Compared to these examples, key states had relatively sparse information. Minnesota, decided by a mere 1.5 percentage points, saw only two new polls fielded in the month before the election, out of five total; Wisconsin, one of Clinton’s most costly losses, had been home to a dozen surveys since October, just as many as Utah.
Such inconsistencies in data no doubt affected analysts’ assessments of the campaign. In the case of North Carolina, 51 total polls painted a slim Trump win and earned the state a Tilting Democratic rating from Inside Elections. Contrarily, 19 polls from Wisconsin, also Tilting Democratic, suggested a 6.5 percentage point Clinton victory. These could be considered polling misses in and of themselves, but re-assigning and balancing a few accurate surveys would have filled gaps across the 2016 map, particularly in the upper Midwest, thereby giving election-watchers a clearer idea of the major states in contention.
Looking ahead to 2020, there are 180 polls across the FiveThirtyEight and RealClearPolitics databases asking about a hypothetical general election between Trump and Biden, as of Monday. Five of the seven states with at least 10 polls — Michigan (15), Wisconsin (14), North Carolina (13), Arizona (13), and Florida (12) — are currently Toss-Ups or Tilting Democrat/Republican. Pennsylvania, another crucial state rated Tilting Democratic, follows closely with eight surveys.
Of course, most of today’s general election findings are tacked-on to existing questionnaires about the Democratic primary, meaning pollsters aren’t yet devoting their full attention to November: in other words, counts are still early. Nevertheless, hopefully these trends will continue on a similar trajectory and focus on battlegrounds overlooked in 2016.
More polls equal more certainty. And as a byproduct of the Electoral College, the uncertainty permeates at the statewide, not national, level. High-quality, recent data plays a crucial role in determining whether we end up on the wrong side of that variability; thus, pollsters should strategically field surveys, and analysts must use this data judiciously.
In terms of polling performance, the 2016 and 2020 Michigan Democratic primaries could not be starker. But by heeding history’s advice and trusting the polls, hopefully a larger number of state polls, a similar error might be avoided come November.