How to Push Phony Poll Numbers in the Public Square

Deep-pocketed companies can get the poll results they want, which is a good reason not to believe industry-supported polls

A Vanderbilt University poll has shown that regardless of political affiliation, Tennesseeans strongly support returning Sudafed and other cold medications that contain pseudoephedrine (PSE) to prescription-only status. This policy has been proven in other states to virtually eliminate meth labs, so it’s unsurprising that Tennesseeans would endorse it after years of suffering from the fires, explosions, burns, poisonings, destroyed property and lost tax dollars that the labs cause. Yet State Senate Speaker Ron Ramsey was shocked by the poll’s result. To understand why, one has to appreciate how a deep-pocketed special interest group can finance polls that create a false impression about what voters really want.

The Consumer Healthcare Products Association (CHPA) represents the manufacturers of pseudoephedrine-containing cold medications that are easily converted to meth. Unlike the pharmaceutical companies that produce the same medications in extraction-resistant formulations, the manufacturers represented by CHPA take in hundreds of millions of dollars a year from meth cooks. As documented by journalist Jonah Engle, CHPA’s desire to protect this enormous revenue stream has led it to spend unprecedented amounts of money lobbying state legislatures to not return PSE-containing medications to prescription-only status.

But CHPA doesn’t just pursue its interests through lobbying. It also releases polls that overstate the public’s agreement with CHPA’s corporate clients. That’s why Speaker Ramsey was “amazed” to find that Democrats, Independents and Republicans (including 64% of Tea Party Republicans) in his state overwhelmingly support returning PSE-containing cold medications to prescription only status. Ramsey, like many other people, had been misled by a press release about a CHPA-sponsored poll claiming that 56% of Tennesseans were opposed to the policy.

CHPA released similar polls in Oregon and Mississippi prior to those states returning PSE-containing cold medications to prescription-only status. In both cases, the policy has been popular with voters and no legislator has been voted out of office for supporting it. The Tennessee poll and others like it are thus a continuation of a well-established corporate strategy of spreading misinformation about public preferences.

Many people assume that wealthy interest groups generate phony poll results by hiring completely dishonest pollsters. That does sometimes happen, but it isn’t necessary for the production of a deceptive polling result.

Poll design involves many subjective decisions about sampling, wording of questions, and order of questions, all of which can change the results. When a special interest group is paying the bills, it has significant power over a poll’s results because it gets to tell the pollster how the poll’s central question should be framed. For example, if a chemical company is facing restrictions on its abilities to dump toxic waste into the water supply, it can commission a pollster to “find out whether the public favors job-destroying constraints on responsible businesses” and release the likely results as “proof” that the public would cheer chemical dumping.

Also, interest groups like CHPA have no legal requirement to release a poll they have conducted. They can legally commission a half dozen polls with different polling methods and release only the one that gives the most favorable result. Neither must they reveal set-up questions and descriptive text that skew answers on the central question of interest (here is a humorous demonstration of how this is done). Neither are they under any obligation to release the data and full list of questions to outside scrutiny. Rather they can legally put only their preferred subset of polling details and results in a press release and maintain that the rest are proprietary information.

That’s how in a state like Tennessee where almost 70% of the public support a policy, a well-heeled interest group can release a poll allegedly proving that 56% of that same public oppose it. Meanwhile, in another state being decimated by meth labs — West Virginia — CHPA is claiming 56% support of its views (yes, there’s that number again) based on the poll it paid for. Given the Tennessee experience, no reasonable person should believe that poll’s result truly reflects the will of West Virginia voters.

Author: Keith Humphreys

Keith Humphreys is the Esther Ting Memorial Professor of Psychiatry at Stanford University and an Honorary Professor of Psychiatry at Kings College London. His research, teaching and writing have focused on addictive disorders, self-help organizations (e.g., breast cancer support groups, Alcoholics Anonymous), evaluation research methods, and public policy related to health care, mental illness, veterans, drugs, crime and correctional systems. Professor Humphreys' over 300 scholarly articles, monographs and books have been cited over thirteen thousand times by scientific colleagues. He is a regular contributor to Washington Post and has also written for the New York Times, Wall Street Journal, Washington Monthly, San Francisco Chronicle, The Guardian (UK), The Telegraph (UK), Times Higher Education (UK), Crossbow (UK) and other media outlets.

39 thoughts on “How to Push Phony Poll Numbers in the Public Square”

  1. I suspect you could get either result depending on how you worded the question assuming you instruct pollsters to push for a yes/no answer or throw out undecided results. I doubt a majority polled know the major evidence in either direction… specifically: (1) pseudoephedrine is a precursor in meth production, (2) cold medication with alternative formulations is either much less effective or totally ineffective (phenylephrine) or much more expensive (meds still on patent).

    In the case of the Vanderbilt poll, the question is: Some over-the-counter cold medicines, such as Sudafed, contain an ingredient called pseudoephedrine that is used to make the illegal drug called “meth.” The Tennessee state legislature recently considered making pseudoephedrine a prescription drug to help reduce meth abuse. Do you support or oppose requiring people in Tennessee to get a doctor’s prescription for pseudoephedrine drugs?

    I think a fairly worded prompt would somehow note that Sudafed is safer/cheaper/more effective compared to other options for most people. Unless some new medicine appeared in the year and a half since I last got a cold in the States, there’s a non-meth-profiteering case to be made for OTC Sudafed.

  2. It’s an example of the general rule: “If you’re not paying for the product, YOU are the product.” In any case where you’re seeing poll results, and you didn’t pay to see them, you should assume that they’re meant to influence your opinion, and may have been the result of one form of manipulation or another.

    I’m amused, by the way, at how you assume the 70% number you cite, is the straight goods, not a result of exactly the same process.

    1. Polls from academic institutions are among the most trustworthy because pollsters there lack the incentives for misrepresentation that industry groups have. Not only are they more at risk of reputational damage (which is a huge deal in the polling business), they also lack the funds to re-run polls till they get the results they want. There is, in other words, good reason to take the 70% as legit and reject the 56%.

      A general point about polls, with respect to the ‘you’re the product’ principle (which is a great one to keep in mind in the internet age), is that you’re really not. Most polling firms do big headline polls as loss-leaders to raise their profile. They have every incentive to do them right, since in releasing them, they’re essentially advertising themselves as capable survey researchers.

      1. Or advertising themselves as good at generating the sort of BS polls wanted by their target market.

        As for the academic institutions, I would remind you of Belesilles getting the Bancroft award after having been exposed as a fraud. After. And some parts of his fraud were rather crude, even a layman with an interest in the subject couldn’t get five pages into Arming America without experiencing WTF moments.

        On subjects with political saliency, where the academic institutions lack sufficient internal diversity of opinion to make sure somebody in the loop would have a different opinion, I believe academic institutions are every bit as capable of exhibiting bias as any other sort of institution.

        Finally, I have a general distrust of polling, based on the observation that people have a tendency to claim whatever opinion they think helps them blend in, if asked about a subject they have no fixed opinion on. Even if polls are reliably measuring something, you’re not guaranteed that what they’re measuring is opinions.

  3. Question: Have people in Tennessee stopped using meth? If not, I think the proper way to put it, is that the policy has successfully relocated meth labs. Potentially, by taking it nation-wide, they could be relocated out of the country, to Mexico perhaps.

  4. I’m against returning pseudoephedrine to prescription-only status, but if that happened, I’d call my doctor and have a prescription within minutes. With my health insurance, it would probably be cheaper than what I pay now (already cheap) Those who lack a family doctor, or who don’t have a good relationship with their family doctor, would suffer.

    1. Unfortunately, the hospital with which my PCP is affiliated will not allow doctors to write a prescription without face-to-face interaction, which would mean another $20 copay for me and about a $60-120 payment by my health plan. Now of course unless PSE was treated like a controlled substance (no refills on scrips, one-time fill only), I could ask for, oh, 11 refills to take care of my needs for the next year, so I guess that would be tolerable.

      1. Whether you have to see a doctor every time or just once is determined by where a drug is scheduled. For PSE, you are probably looking at schedule III/IV so someone with allergies who took it all the time would have to visit a ftf doc once and then could get the script renewed by phone.

        Of course this is a problem for people who never have a single doctor visit in the lifetime, but I don’t think that population’s primary struggle as being a lack of PSE — it’s a much bigger problem than that.

        1. Or, you could make this extraction-resistant widely version available OTC *before* you make it otherwise-prescription only.

          That would be another way to go about this. If one cared about allergy sufferers.

          1. They are already available, so you don’t need to do anything to make them so. All that would be required (see embedded links) is to make the prescription requirement apply only to the non-resistant versions.

  5. Keith is talking about one form of selective outcome reporting, a major source of bias in research of all kinds. It gets especially serious when drug companies are allowed to be selective about the reporting and publishing of clinical trials of their products with short-term data, surrogate outcomes, and other trickery which arises from their having custody of the study data.

    The CHPA is only one example of how commercial interests can influence what is reported as science.

    1. This is a reasonable objection which warrants a reply.

      There is actually some empirical evidence that drug companies report outcomes selectively in their published trials. In 2009, an article (should be free access at, in the New England Journal examined internal documents of clinical trials of gabapentin from Pfizer and Parke-Davis, comparing the reported outcomes with the outcomes specified in the original study protocols. Key quote from the abstract: “The primary outcome was changed in the case of 5 of 8 published trials for which statistically significant differences favoring gabapentin were reported. Of the 21 primary outcomes described in the protocols of the published trials, 6 were not reported at all and 4 were reported as secondary outcomes. Of 28 primary outcomes described in the published reports, 12 were newly introduced.”

      There is also an abundant literature on the fact that clinical trials sponsored by industry find favorable results for their products more often than in trials not sponsored by industry. The gabapentin article has the advantage of being free and also pertaining directly to the issue of selective outcome reporting.

      One amusing example was a 1994 study of industry-sponsored randomized trials of NSAIDS (drugs like ibuprofen and naproxen) in which manufacturers compared their products with other NSAIDs. For the published trials, all of the sponsors’ products were as good as (71.4%) or better than (28.6%) the comparison drugs and were also less toxic (86.4%). Like the children of Lake Wobegon, all of the sponsors’ NSAIDS were above average. The abstract is but the full text is not online.

      1. This was the main rationale for requiring trial registration, as selective reporting is so common.

        1. Clinical study registries since 2005 have had some success but have been far from perfect. For high impact journals publishing studies of interventions for gastroenterology, cardiology, and rheumatology, a large proportion of trials indexed in 2008 were registered (Mathieu et al, JAMA. 2009;302(9):977-984 ); 147 of 323 (45.5%) of trials were adequately registered, having been registered in advance with primary outcomes specified; however, 46 of these 147 trials showed some evidence of discrepancies between the outcomes registered and the outcomes published . The track record for interventions like physical therapy (published mostly in low impact journals) is much worse (Pinto et al, Phys Ther. 2013; 93:299-309.) Selective outcome reporting was present in about half of these studies published in 2009, and only 5% were prospectively registered with unambiguous primary outcomes.

          Clinical trial data should be seen as a public good and not as proprietary information; the same should hold true of polling data like the kind that started this thread. Transparency should not be an impossible goal.

      2. This commenter has been banned for violating the guidelines laid out above. Bans cover all commenting in perpetuity, whether substantive or not.

  6. I like to keep a set of extra knickers in a drawer in my office so that from time to time I can take them out and twist them pointlessly. They also work for getting my undies all in a bunch when that is called for.

  7. “Tennesseeans strongly support returning Sudafed … to prescription-only status.”

    Was it at one time prescription-only in Tenn.?

        1. FDA made the change in 1976. Whether or not that was due to industry pressure, I do not know.

  8. PSE has legitimate therapeutic value and is also subject to abuse. This is true of countless controlled substances. Making PSE a prescripton medicine is a mundane example of a general practice in federal drug law. I personally would rather have it behind the counter with a requirement of photo ID sufficient to purchase a reasonable supply, given that non-recreational users may encounter barriers to access which could be lowered, with the dispensed dose being a sufficiently small fraction of the amount needed to manufacture meth that it does present an unreasonable barrier for illegitimate use.

    1. This seems to the policy here in MA. Presumably that is state and not federal law. Or is it federal?

      1. It’s the policy here, too, in SC.

        It seems to me this measure is entirely sufficient to attain the desired level of inconvenience to meth producers, without forcing allergy sufferers to pay a doctor for the privilege of taking what should, by medical standards, remain an OTC medication. We’re routing too much of medicine through the guild of medical gatekeepers as it is.

      2. This is how it’s done in VA. Have to go to the back counter, show driver license, sign a register, and you only get one (two?) at a time. Seems a reasonable way to do it.

        1. Based on the fact that anything more than requiring somebody to sign for their pseudophed would seriously inconvenience allergy sufferers, and relocating meth labs isn’t an important enough goal to justify more than slight inconvenience.

        2. @byomtov: It is a federal law, the CMEA of 2005.
          @Brett: Because they are not backed by any evidence, your arguments are not compelling.

  9. Oregon did virtually eliminate meth labs after making PSE prescription-only. We need comparisons of meth lab curtailment in states which made it behind-the-counter. shows “meth incidents” in all 50 states between 2004 and 2012. Oregon incidents did indeed decline from 632 in 2004 to only 11 in 2011 and 9 in 2012. Colorado, which has behind-the-counter access to PSE, decreased the number of incidents from 411 in 2004 to 13 in 2011 and 14 in 2012. Alaska went from 121 in 2004 to 5 in 2011 and 1 in 2012. It appears that many states with behind-the-counter PSE curtailed meth labs significantly between 2004 and 2012.

    Tennessee appears not to have done as well. It went from 2341 in 2004 to 2315 in 2011 and 1585 in 2012; Missouri went from 2913 in 2004 to 2015 in 2011 and 1825 in 2012. It is not surprising that its citizens would want a more reliable law to eliminate a social pest, and are not satisfied with the measures which have seemed to have an effect in other states. Mississippi stayed about the same for years until its prescription-only law went into effect; it went from 321 in 2011 to 5 in 2012.

    It is almost certain that there is a strong interaction between regional geography (the Mountain West vs Appalachia/Midwest) and the effect of behind-the-counter laws for access to PSE.

    This is where some social science researcher in search of a thesis or dissertation topic will say, “Hmmm. Wonder why that is.”

    1. @Ed Whitney: What you will miss if you just look at 2004 and 2011 or 2012 is the number of states that dipped sharply in 2005-2006 and then came back as smurfing eveolved to subvert the ID checks.

      1. The smurfing rebound is apparent in some states but the Mountain West and West Coast appear to be monotonic decreasing from 2004 to 2012. Still very curious.

        If I were in search of a dissertation topic, I would be tempted to see if the limits on pharmacist dispensing were the same in all of the states which moved PSE behind the counter. State-to-state variations in these limits, if present and correlated with smurfing rebound, could shed light on the differences in regional patterns.

  10. Brett’s point about merely relocating the labs has merit, methinks.

    Having them behind the counter w/photo ID requirement + maximum purchase restriction makes sense to me. Some inconvenience is ok, if it really does clean up the backyard, as it were.

      1. Let’s be clear: Are you asserting that domestic meth labs serve an entirely different market than foreign meth labs exporting to the US, so that eliminating domestic meth labs will result in NONE of the users they formerly supplied switching to imported meth?

        Because, that’s the implication of claiming that these measures do not relocate meth production.

        1. You are making a different argument now. You originally said the labs would be relocated, which is different than arguing that large pre-existing labs may produce more meth to make up for the disappearance of small labs. The former assertion has no evidence behind it, the latter is entirely plausible. In any event, the purpose of the policy isn’t to reduce meth use or production, but to reduce production-related harms, which are concentrated in small-time labs for obvious reasons.

          1. Granted, that last, which is the only reason I’d support the proposal at all. Because the only harm from drugs I actually care about is the harm to third parties, who haven’t chosen to undertake the risk. People are entitled to fry their own brains, but small scale meth labs harm people who haven’t voluntarily assumed the risks. And so they’re worth doing something about, even from a libertarian perspective.

  11. Even though I don’t like the idea of making people go to a doctor just to get PSE — and btw, why the sympathy for cold sufferers and not people with allergies? it feels the same! it is not a mere inconvenience — I agree that lying liars are **bad.** And being selective, or slanting things … it’s all a version of being a lying liar and it’s wrong.

    So, I agree that twisting polls is wrong. Frankly I kind of agree with the person who said they are all basically suspect. Who has time to find out how they asked the questions? When we figure out a group that debates scientific study validity, and summarizes it for me a la Consumer Reports, we can make a subgroup who will do this, too. Full employment for scientists!!! Hear, hear.

Comments are closed.