Only McAulliffe’s Polling was Accurate in Virginia

Why was Terry McAuliffe the only candidate in the Virginia Governor’s race whose support was well-predicted by polls?

The closeness of the Virgina Governor’s race surprised many political observers. To understand why, take a look at this table, which was created from Real Clear Politics’ helpful summary of the eight polls conducted in the week leading up to election day (October 30-November 5). The top line shows the average (not weighted for poll sample size) performance of the three candidates in the eight polls, including the “don’t know” response option. The second line drops the “don’t knows” and reports how much support each candidate had among those poll respondents who expressed a preference. The third line is the actual election day result. McAuliffe is the only one who performed pretty much as expected.

Virginia

Sarvis’ support was grossly overstated in closing week polls by a factor of about 1.5 to 1. I am comfortable calling this eminently predictable because I predicted it. It’s simply mathematically harder to predict rare events (e.g., third party votes) than common events; even a few days before the election one poll had Sarvis at double the level of support he actually received.

Less predictable and therefore more intriguing is how Cuccinelli outperformed his polls, whether you analyze them in the aggregate as does the table or individually (none of the eight had him as high as his actual total). One could speculate endlessly about why this happened, but I am partial to Chris Matthews‘ view of polling: If pollsters sounded like Archie Bunker instead of staid professionals, there’s a slice of the electorate who would be more forthcoming about their less-than-enchanting political views.

Author: Keith Humphreys

Keith Humphreys is the Esther Ting Memorial Professor of Psychiatry at Stanford University and an Honorary Professor of Psychiatry at Kings College Lonon. His research, teaching and writing have focused on addictive disorders, self-help organizations (e.g., breast cancer support groups, Alcoholics Anonymous), evaluation research methods, and public policy related to health care, mental illness, veterans, drugs, crime and correctional systems. Professor Humphreys' over 300 scholarly articles, monographs and books have been cited over ten thousand times by scientific colleagues. He is a regular contributor to Washington Post and has also written for the New York Times, Wall Street Journal, Washington Monthly, San Francisco Chronicle, The Guardian (UK), The Telegraph (UK), Times Higher Education (UK), Crossbow (UK) and other media outlets.

20 thoughts on “Only McAulliffe’s Polling was Accurate in Virginia”

  1. From these numbers, it looks like differential turnout among the respective bases – has anybody got numbers? This is worrying for the Dems going into the 2014 election. The political lesson of the last four years is that winning the Presidency is not enough.
    Daily Kos blogger wdrath:

    As far as Obenshain, it appears that he and Cuccinelli were able to steal the Attorney General’s office, by deliberately, with malice aforethought, trying to disenfranchise as many likely Democratic voters as they could, by “purging” voters from the voting rolls just a couple of months before an election.

    Voter suppression doesn’t work with big margins like McAuliffe’s and Obama’s, but it’s effective in enough close races for the tactic to be a good deal for the GOP.

    1. No question that’s a factor, but it’s overwhelmed by Gov-Elect McAuliffe’s character. I don’t think the Democratic Party will be able to find a slate of candidates as lame as him for 2014.

        1. While the Democrats may search extensively, the GOP will select their losing candidate without benefit of a primary again and find Cooch-2.

    2. Sure, Presidency is not enough. But this was an off / off year (no congress / Senate races), and the standard bearer was McAuliffe. Obama got 2 million votes in 2008; Mac only got a bit more than 1 million for the win.
      So if the Dems pick a good Prez candidate – and the Rs pick a tea party loon in ’16, it will be a real Dem boon. Lots of what-ifs in there, though.
      The real challenge is good turnout in 2014…everyone hates Congress, but is ok with their own Congressman. With VA’s redistricting, even with a majority of Dem votes they’ll still return a R slate.

      1. It is true that the slogan “throw the rascals out!” usually has a hidden opinion, “but my rascals isn’t so bad” embedded. That may not be the case this time. I heard a story on NPR this morning that the Pew Center poll found that nearly 2/3 of those polled want to throw their rascal out of Congress. Per NPR, this is the first time ever that number has been nearly so high.

        Disclaimers:
        1. I haven’t seen any crosstabs on this, so I’ve no idea how the opinions break across the D/R chasm.
        2. It’s crucial how that breaks. If the opinion is nearly independent of voter affiliation, this is extremely bad news for any incumbent.

  2. Isn’t this predictability more a function of the rules of the game than of a universal mathematical inevitability? In this case, we essentially have a 3-sided coin that’s weighted in favor of two sides. If the rules of the game had been, say ‘instant run-off voting’ instead of ‘first past the post’ I suspect the first round tally would have been unsurprisingly closer to the averages. Quite possibly, it may have tipped the election the other way — and this would have been entirely unsurprising by factoring in the voter’s 1st and 2nd preferences.

    When the game is ‘fair’ and the players understand that => high predictability; when the game is ‘unfair’ but the players believe it’s fair => lower predictability (unless you can factor in the unfairness).

    1. @Tim: That’s an intriguing hypotheses that I suspect has been tested in political science studies, although I don’t know that literature. It is a non-competing hypothesis to the difficulty in predicting rare events versus common ones, which is mathematical reality that is always in force. See this post for the details and a worked though example.

      http://www.samefacts.com/2012/07/polling/why-polls-of-third-party-candidate-support-are-usually-wrong/

  3. I’m not sure what you mean by “If pollsters sounded like Archie Bunker instead of staid professionals…” Interviewers are generally paid low wages, so there are surely plenty who do not sound like “staid professionals.” When I worked in opinion research (briefly and long ago), our corporate office was in an upper-middle-class suburb and staffed by well educated, predominately white employees, but the call center was in a nearby poor, predominantly black city, and the interviewers were quite diverse (by age and by race).

    I have no idea of what the race, ethnicity, education level, or social class of the interviewers for these polls were, much less to what extent those characteristics would be identifiable to respondents, or the extent to which they would affect the responses. (Fortunately, the standards for blog comments are pretty low.) But a 1989 study of Virgina polling (http://www.pitt.edu/~finkel/download/race.pdf) found “a race-of-interviewer effect on the vote intention of white voters of 8-11 percentage points.” This was 24 years ago and one of the candidates was black, so the magnitude would have been much greater than in the 2013 race, but it suggests that the distinction might not be “Archie Bunker” vs. “staid professionals.” Plus Archie Bunker had a New Yawk accent, not a southern one.

    I’d definitely think that the responses would differ between, say, an interviewer who sounded like Boss Hawg and one who sounded like Al Sharpton.

    1. I think Keith’s point was more about the “coziness” vs “political correctness” factor in getting some people to indicate their true preference, rather than in poll questioners trying to act as a specific character.

      The whole basis of anxieties often derided as the imposition of “political correctness” is that one’s true racial animosities might somehow be revealed without any actual use of the “n” word or wearing of pointy-headed costumage. Most racism these days is carefully nurtured and concealed, yet remains just as active as ever in determining one’s view of the social order. I doubt that things have changed much from the cited 1989 study, given the steady hardening of views on race among many conservatives as the courts have given them encouragement that racial disparity is no longer a factor the government need seriously address.

      1. @npm: Plus Archie Bunker had a New Yawk accent, not a southern one.

        You are taking me too literally. The point is that social cues may lead people to be more or less comfortable in revealing preferences which they know are socially disapproved of. It may take a proactive cue to get people to reveal particularly unsavory views, and pollsters are generally trained not to offer such cues.

        1. Exactly.
          l
          Then you have push-polling, when that effect is sometimes intentionally invoked. There the accuracy of the polling isn’t the issue it is for real pollsters, rather the goal of influencing voters to either act on their views or to discourage them from doing so. Completely different goals, but the the voter picking up to listen to them often has a hard time determining the difference.

  4. This is sort of a reverse Bradley effect, which towards the last days of the 2008 election was practically John McCain’s only chance of outperforming Obama on poll days. In that scenario, respondents would tell pollsters that they’d vote for an African-American candidate (e.g. LA mayor Tom Bradley), but then on election day they’d change their mind. While that may have been the case for Tom Bradley, comparing Obama’s election results v. polling numbers didn’t reveal the kind of discrepancy needed to substantiate the hypothesis.

    Could we be over-estimating the self-shaming and/or self-doubt and/or cultural insecurity by respondents, especially when responding via phone? If people really do feel so embarrassed to admit their own political preferences to someone so anonymous and disinterested, maybe we should be nominating different candidates.

  5. Isn’t it fairly common for people polled as supporting Libertarian candidates to go for the Republican in the actual event?

    Ron Paul was stumping for Cuccinelli with lots of nullificationist rhetoric at the last minute. That might have swung some of the don’t-tread-on-me types from Sarvis to Cuccinelli. But Libertarians usually poll better than they actually do.

    1. […]
      Finally, while it didn’t change the outcome, the third-party candidate in the race, Libertarian Robert Sarvis, may have made it closer for McAuliffe than it would have been otherwise. Had he not been on the ballot, a third of his voters said they’d have supported McAuliffe – slightly more than twice as many as said they’d have gone for Cuccinelli.

      Exit Poll Results Tell a Tale of Two Republicans

      Of course, if Paul’s endorsement of Cuccinelli drew votes from Sarvis, the voters who did vote for Sarvis would more likely be people who wouldn’t have voted for Cuccinelli if Sarvis hadn’t been in the race.

  6. Also, note, if Cuccinelli’s added support came from former Sarvis supporters, that’s hardly a Bradleyesque case of people hiding a socially-disapproved-of response; if anything minor-party candidates tend to have a worse reputation as cranks or losers.

  7. As some of us pointed out in previous rounds, your statistical analysis was entirely incorrect. That’s a mathematical fact, not a matter of opinion. The explanation for this particular result, including both the drop in the 3d party vote and the rise in the closer of the two-party votes, is entirely non-statistical, simple, well-known, and explicitly “predicted” by several of us. People want to vote for a third party, express that view when polled, but then switch to one of the candidates that might win and holds views not too far from their own.

Comments are closed.