What is policy analysis?

Debates over public policy are full of what Ambrose Bierce called “vagrant opinions, without visible means of support.” Those of us who try to do policy analysis as a serious intellectual pursuit continually face the charge that the analysis we present is merely a fig-leaf to hide our otherwise naked interest or prejudice. And of course advocates for specific policies (or the interests they are designed to serve) have learned to dress up their lobbying efforts in analytic clothing.

When I teach courses in the methods of policy analysis, my first problem is to try to convince students that such a topic exists, without inducing the false belief that there exists some set of methods which, if properly applied, can provide “the right answer” to any problem.

Here’s the introductory text from that syllabus:

Public policy analysis is the discipline that considers what actions would best serve the public interest in a given situation, and how those actions can be implemented through actual institutions. Like any discipline, it seeks the truth, but the truth sought is pragmatic, in the original sense of that term: truth about what is to be done, rather than merely about how the world is to be described.  That differentiates policy analysis from social science.

Policy analysis differs from policy advocacy in that it regards policy choices as instrumental: it reaches a policy conclusion at the end of its argument, rather than starting with a preferred policy and seeking arguments in its support.

A full policy analysis, if one were feasible, would answer the following questions:

1. What is the situation to be considered?

2. What suggests that this situation calls for a specific policy intervention, rather than being left to individual choices, markets, and ordinary formal and informal social controls?

3. What is at stake? What are the positively or negatively valuable outcomes of alternative policies, or of inaction?

4. On what basis should those outcome dimensions be weighed against one another?

5. What specific options exist, or can be invented? (Your answer to #2 ought to provide some hints. Remember that “doing nothing” and “continuing current policies” are always options.)

6. What is the probability distribution of the outcomes of each option on each of the identified outcome dimensions?

7. Which option has the best distribution of possible outcomes, appropriately weighted?

8. What actions would need to be undertaken to implement that option? What agencies exist that could undertake such actions? What changes would need to be made in those agencies if they were to do so? Are those changes feasible, and if so what would the costs be of bringing them about? Would the agencies in question then be more or less capable of producing public value in other ways?

9. Alternatively, what new institutions would need to be created, and what is the likelihood of doing so successfully? What would be the side-effects of creating such agencies?

10. What are the predictable implementation problems, both operational and political? How can they be overcome, or at least ameliorated?

11. What decision-makers would need to agree to the proposed course of action? What are the prospects of their doing so?

12. What social and political dynamics would the proposed policy set in motion? In particular, would the policy itself evolve appropriately over time, or would it instead tend either to degrade toward a less desirable policy or to fossilize as the conditions that make it desirable now change?

13. Are there second-best options with greater administrative or political feasibility, or more desirable likely evolutionary paths, than the chosen policy?

Any actual analysis will fall short of giving a full answer to those questions, and is to be judged, among other things, by how carefully it indicates the questions it has omitted and the associated ranges of uncertainty.

Notice that item #4 involves the application of some set of values, and that, as as result, people with different values will often reach different conclusions even if they agree on the facts. Benefit-cost analysis, which uses the willingness-to-pay principle, is a useful heuristic for many policy choices, but applying it uncritically means accepting the existing distribution of income and wealth as optimal and assuming that what people (say or think) they want is invariably identical with what’s actually good for them.

So we should not expect the application of policy-analytic methods to lead everyone to agree. What we should expect it to do is to eliminate options that don’t make sense under any reasonable set of values, and to help identify the factual and normative bases of actual disagreements. It also forces it practitioners, at least while they are doing it, to step back from individual or group self-interest, and from received opinion and factional loyalty, and try to reason in the public interest.

Author: Mark Kleiman

Professor of Public Policy at the NYU Marron Institute for Urban Management and editor of the Journal of Drug Policy Analysis. Teaches about the methods of policy analysis about drug abuse control and crime control policy, working out the implications of two principles: that swift and certain sanctions don't have to be severe to be effective, and that well-designed threats usually don't have to be carried out. Books: Drugs and Drug Policy: What Everyone Needs to Know (with Jonathan Caulkins and Angela Hawken) When Brute Force Fails: How to Have Less Crime and Less Punishment (Princeton, 2009; named one of the "books of the year" by The Economist Against Excess: Drug Policy for Results (Basic, 1993) Marijuana: Costs of Abuse, Costs of Control (Greenwood, 1989) UCLA Homepage Curriculum Vitae Contact: Markarkleiman-at-gmail.com

4 thoughts on “What is policy analysis?”

  1. "4. On what basis should those outcome dimensions be weighed against one another?"
    The frame is hard consequentialism. I would agree that a deontology that ignores consequences is insane, but a consequentialism that ignores deontological constraints (as a minimum as social facts) is not going to work.

    Another way of looking at this is that you need to evaluate inputs as well as outcomes. Clearly, many Americans have strong priors against new taxes and regulations. Many everywhere have strong priors in favour of entrenched fundamental rights. These are possible costs of an interventionist policy. It's possible to redefine "outcome" as "anything that changes", which seems artificial.

    John Quiggin has made a case for using opportunity cost as a guiding paradigm for thinking about economic policy. I'm only half convinced, but the concept is powerful and deserves consideration.

  2. You may want to consider asking your students on the first day of class to write an answer to the question: What is "policy?" As a consulting policy analyst, I worked with multiple state government agencies in California to assist staff in writing policy manuals for their operations. I found it useful to begin that process by finding out what they think "policy" is. Many implications for implementation flow from differences in understanding of that word, which many people are surprised to discover does not have a universally agreed-upon definition.

    1. In my life as an international civil servant in a bilingual organisation, one long-standing problem was how to translate "policy" into French. Politique covers both policy and politics. You can always try action gouvernementale. But I have wondered whether there is a deeper conceptual divide. In the English-speaking world, policy is Platonically pure and politics the grubby sausage factory. In principle you could have policy without politics, as with Palmerston's concept of foreign policy as the expression of permanent and objective national interests. The lack of a distinction in French reflects an idealisation of politics, or at least a recognition of its necessity.

  3. I find that a lot of the game tends to be in Question 1, what is the situation/problem we are trying to fix/solve? A very important part which is often neglected, though you can also get bogged down there. And of course, "public interest" itself is always worth a think.

    Meanwhile, many non-practitioners, particularly lawyers ime, tend to think "policy" just means half-baked economic efficiency arguments. No pretense of caring about public interest is made.

Comments are closed.