Dealing with Adverse Opinion Polls

The team at Number 10 desperately do not want to hold a referendum on whether the UK should stay in the EU. Yet they are faced with a series of polls (like this one) seeming to show that the public strongly desires such a vote.

Here’s how Sir Humphrey would have handled it:

Author: Keith Humphreys

Keith Humphreys is the Esther Ting Memorial Professor of Psychiatry at Stanford University and an Honorary Professor of Psychiatry at Kings College London. His research, teaching and writing have focused on addictive disorders, self-help organizations (e.g., breast cancer support groups, Alcoholics Anonymous), evaluation research methods, and public policy related to health care, mental illness, veterans, drugs, crime and correctional systems. Professor Humphreys' over 300 scholarly articles, monographs and books have been cited over thirteen thousand times by scientific colleagues. He is a regular contributor to Washington Post and has also written for the New York Times, Wall Street Journal, Washington Monthly, San Francisco Chronicle, The Guardian (UK), The Telegraph (UK), Times Higher Education (UK), Crossbow (UK) and other media outlets.

7 thoughts on “Dealing with Adverse Opinion Polls”

  1. I can’t thank you enough for this clip — it’s going straight into the file for use in this fall’s Survey Research Methods course. Context effects explained in 2 minutes flat!

  2. Snark aside, I’m not sure if the British electorate fully understands the consequences of leaving the EU.

    Basically, there are two options that the UK would have. One would be to leave the EU, but stay within the EEA. The other would be to leave both the EU and the EEA.

    Staying within the EEA is not necessarily a good thing. EEA member countries still have an obligation to implement almost all EU directives and regulations relating to the internal market (the big exception being agriculture/fishing), but have no representation in the EU institutions and bodies. Great Britain would, in particular, lose its representatives the European Parliament, and more critically, its vote in the Council of the European Union and thus any power to veto EU legislation.

    Leaving the EEA as well would most likely be economically disastrous. For better or worse, more than half of the UK’s trade occurs with the EU. The free trade is also rather important for the UK’s outsized financial sector. Leaving the EU/EEA would also make the UK more subject to pressures from third countries, especially the world’s superpowers (the growing strength of the EU has famously — and unhappily, for the US — made it much more resilient against US attempts to impose its will on EU member states).

    Neither option is very attractive to the UK government, for obvious reasons (Labour and Liberal Democrats also tend to be in favor of EU membership as a matter of principle). But the Murdoch papers have been whipping up a frenzy (their favorite recent target has been ECJ human rights jurisprudence), which, combined with traditional Tory euroscepticism has had a deteriorating effect on public opinion.

    I wonder what would happen if a polling organization, rather than conducting a push poll along the principles laid out by Sir Humphrey, conducted one that clearly explained the alternatives and its consequences.

    1. The interesting thing — and I don’t entirely understand it — is that many people view clear explanations of alternatives and consequences as push polling.

      A few years ago here in New Mexico we had a domestic partnership bill before our legislature (SB 12). Leaving aside how quaint the idea of domestic partnerships seem today, SB 12 was the first time it was believed the issue had even a remote chance of passage. Our Senate had passed the issue previously, but the House balked every time. This time, it looked like the House was going to pass it.

      Needless to say, the Archbishops in New Mexico threw a shoe. SB 12 failed to get out the Judiciary Committee when one Democratic senator “had to take an important phone call” (during the vote) and another sided with the Republican minority to generate a tied vote. Senator Phone Call later said she would have voted against a “Do pass” recommendation, because that was what her constituents wanted. Senator No! said he was personally okay with it, but his constituents weren’t.

      So the ACLU and Lambda Legal Defense (and others) put up some money to poll the registered voters in their districts. The survey clearly described SB 12 and its effects. The surveys showed near 60% favored or strongly favored passage.

      I use this as a teaching example — anecdotal data (like phone calls coming into a legislator’s office, or letters, or …) are misleading. But close on to half of my students view the question with its description of the consequences as being a leading question. When asked, “Okay, how should the question be worded?” they are at a loss, except that they don’t like providing information.

      These students are difficult to convince that providing context (and not in the form advocated by Sir Humphrey) is necessary for matters people have not kept informed on.

      1. One problem is that it is often difficult to describe context in a neutral fashion. For example, while for the most part there is a fairly objective description of the consequences of leaving the EU for the EEA (because they’re captured formally in statutes and treaties), it is difficult to even estimate the consequences of a country leaving the common market entirely, let alone describe them neutrally.

  3. Dennis: Your students may not be as naive as you think. I have heard a gazillion presentations in which someone said something like “only 20% of voters supported our proposal, but once we let them know it would help the economy, lower taxes and help them lose weight while they sleep, then 90% were in favour!”

    What is provided in these cases is what someone thinks is information, but if that person/group has a strong view of the issue their “information” may be hortatory rather than neutral. This is testable — have groups with different views and no view provide “information” and see if the content of the information and the response of voters differs.

    1. Where do I sign up? I especially like the lose weight while I sleep part… this dieting and going to the gym thing is tiring.

      Finding a situation where the politicians are actually called when they hide behind their constituents’ skirts is rare in my experience. I wish that the sponsors had something along the line you suggest.

      But perhaps I did view the script as innocuous because I hold views similar to the ACLU on the matter.

Comments are closed.