Survey research under totalitarian rule

Turns out it can be done, at least in Syria.

Survey research is illegal in Syria, unless the survey is conducted by the government. So if you’d asked me about the chances of pulling off an actual survey on questions such as government performance and corruption, and getting something like a representative sample, I would have said, “Somewhere between slim and none.”

Which just proves that even blogging has not succeeded in making me infallible. An outfit called the Democracy Council actually managed to organize more than 1000 face-to-face interviews. They couldn’t do real equal-probability sampling, but they stratified so as to get a pretty reasonable cut of the Syrian population, albeit not gender-balanced.  Angela Hawken of Pepperdine, a methodological Puritan, did the reweighting and has blessed the results. Really, a pretty impressive feat of data-gathering.

The results aren’t very surprising, except for the fact that the respondents don’t seem to have been intimidated. (Also, I didn’t know that Syria is only 3/4 Muslim; more than 11% are Christian, and about the same are “other.” Wonder what “other” is? But that’s just about my ignorance, not an objectively surprising finding.) They have limited internet access, but 97% get satellite TV, so they aren’t entirely information-starved. Civic participation is very low.

By big majorities, Syrians dislike the government, think it’s crooked, want an end to martial law, and are pessimistic about the country’s future but more optimistic about their own future and their families’. If they were to emigrate, Europe and the Gulf are more popular destinations than the U.S.

So the big news from the survey, it seems to me, is that it could be done at all. Accurate information about public opinion in places like Syria might make important differences in policy choices.

Author: Mark Kleiman

Professor of Public Policy at the NYU Marron Institute for Urban Management and editor of the Journal of Drug Policy Analysis. Teaches about the methods of policy analysis about drug abuse control and crime control policy, working out the implications of two principles: that swift and certain sanctions don't have to be severe to be effective, and that well-designed threats usually don't have to be carried out. Books: Drugs and Drug Policy: What Everyone Needs to Know (with Jonathan Caulkins and Angela Hawken) When Brute Force Fails: How to Have Less Crime and Less Punishment (Princeton, 2009; named one of the "books of the year" by The Economist Against Excess: Drug Policy for Results (Basic, 1993) Marijuana: Costs of Abuse, Costs of Control (Greenwood, 1989) UCLA Homepage Curriculum Vitae Contact:

7 thoughts on “Survey research under totalitarian rule”

  1. Both Alawite's and Druze are sufficiently heterodox that they are often considered not to be Muslim at all. It's also the case that the Assads, the ruling dynasty of Syria, are Alawites themselves.

  2. The problem with polling in a repressive society isn't just the possibility that polling is illegal. It is also that people will shade their answers, out of fear that the pollster is actually a government informant. IOW, the sample might be representative, but are the answers honest?

  3. There’s a neat trick (I forget who published it first) where you give the survey respondent a die and two yes-or-no questions, and then say “Roll the die where I can’t see it. If it comes up 1 or 2, tell me your answer to question A. Otherwise, tell me your answer to question B. Don’t tell me which question you are answering or the results of the die roll.” If you already know what proportion of your sample population will answer yes to question A, you can what portion would answer yes to question B (with a higher margin of error, of course).

    The provided link is dead so I don’t know if that’s how they handled the Syria poll.

  4. Seth,

    The link is

    They don't seem to have used randomized response.

    Randomized response (Warner, 1965, JASA; Greenberg, 1969 JASA) is indeed a neat trick, but it has big limitations. You have to have a reasonable idea how common the sensitive behaviour is to get tolerable efficiency, and the subjects need to understand the trick. More importantly, the fact of being chosen for interview or having agreed to be interviewed is often sensitive (probably true here), so the interviewer still needs to be trusted. I think it's more useful for information that is embarrassing, rather than actually dangerous.

  5. All surveys are suspect because any sampling only includes people willing to participate. Of course in this case people unwilling to participate it is probably safe to assume are distrustful of their government.

  6. I heard about a Chinese poll of poor households. They got a 100 percent response rate. Yes, in part because it was mandatory, but they also recognized that taking time to answer a poll was a hardship for people who were just barely surviving, so the pollster also stayed to assist with the household chores that didn't get done during the interview.

Comments are closed.