Principles of Policy Analysis: Final Exam

I’ve just returned from a semester as the Schelling Visiting Professor at the University of Maryland’s public policy school. While there, I taught a course in the principles of policy analysis to some of the second-year students. They were a lively, serious, and curious group of people, and serious about figuring out how to enact the public good. But none of the questions on the final exam (below) covered material they were familiar with at the beginning of the course. For example, they all knew Bayes’s Rule, but only as a trick for proving theorems in statistics, not as a way of integrating new knowledge into a framework of existing beliefs.

I admit that questions 3, 4, 6, and 8, and the second part of question 5, are somewhat idiosyncratic. That is, I think the underlying concepts are highly relevant to doing policy analysis, but not everyone would agree.

On the other hand, 1, 2, the first part of 5, and 7, and all of Part II, seem to me central to the enterprise of thinking clearly about policy choice in a world of scarcity and uncertainty: as central, I would claim, as some of the ideas from statistics and microeconomics that virtually every MPP is familiar with (at least at the time of the final exam). Yet since none of them fits into a first course in statistics, economics, politics, or management, and since public policy education as now practiced mostly consists of studying statistics, economics, politics, and management with statisticians, economists, political scientists, and organizational behavior folks, a basic distinction such as the one between benefit-cost analysis and cost/effectiveness analysis is unlikely to get much attention.

Note to public policy professors, department chairs, and deans:

Could your graduating MPPs pass this exam?

Part I Answer as briefly as possible:

1. What policy question does the benefit-cost calculation allow you to answer that a cost/effectiveness analysis can’t answer?

What evaluative question does a cost/effectiveness analysis allow you to avoid that a benefit-cost analysis requires you to address?

Give an example (with made-up numbers) of a simple benefit-cost analysis and a simple cost/effectiveness analysis applied to the same problem.

2. What inefficiency results if consumers are required to pay in order to use non-rival goods? How is it that consumers could be made worse off if producers of non-rival but potentially excludable goods are not allowed to make them actually excludable (that is, not allowed to deny access to the goods to those who do not pay)? Give an example of each.

3. What would you need to know to determine how much individual social capital (relational assets) a given person has?

4. How does reputational externality create a collective-action problem? Give an example (actual or made-up) that illustrates the nature of the problem.

5. Assume the expected value of the punishment for parking illegally is a $30 fine. If people made decisions about parking based on strictly selfish rational calculations and had diminishing marginal utility of income, would there be more illegal parking if:

a. Everyone who parks illegally is fined $30; or if

b. One illegal parker in 100 is chosen randomly and fined $3000?

Why is that so?

Name two theories about imperfect rationality that, if they explained actual behavior better than the rational-actor theory, would change your conclusion, and briefly explain why.

6. What distinguishes a high discount rate from “temporal myopia,” as predicted for example by hyperbolic discounting? Give an example where you could tell which sort of behavior pattern some individual was displaying.

7. How does risk-spreading trade off against incentive effects? Give an example.

8. How does the theory of conspicuous consumption relate to the idea of market signaling? In what way is the result a market failure? Give an example.

Part II

You are the Director of Public Works in a town with a total annual budget of $3M. You need to build a temporary building to use as a Town Hall while the main Town Hall is reconstructed. Reconstruction will take three years. (Assume, implausibly, that there is no uncertainty about how long it will take.) Once the remodeling is done, the temporary building will be torn down.

A. You have two construction options. Plan A costs $1M to build (including the cost of borrowing the money to build it) and $200k per year to operate. Plan B costs $1.1M to build (again, including financing costs) and $100 k per year to operate. They take the same amount of time to build, and are equally attractive and convenient.

What do you need to know in order to make the decision? What is the critical value of that variable?

B. Now assume you’ve chosen Plan A over Plan B. Someone proposes Plan C instead. It has the same operating costs as Plan A, but the construction cost will be either $900,000 (if the job can be done without sinking pilings) or $1.1 million (if pilings are needed). There’s no way to tell for certain whether pilings are needed until construction has started, after which it would be too costly to switch back to Plan A.

Your geologist says it’s about an even chance whether pilings would be needed or not. Thinking in terms of the interests of the taxpayers, which would you choose, and why?

C. A firm of consulting geologists offers to do a preliminary soil analysis for $20,000. The analysis is not perfect. Its sensitivity (to the need for pilings) is 80%; its specificity is 75%. Based on the results of the test, you can buy actuarially fair insurance that will guarantee a fixed construction cost.

1) Is the test worth the proposed price?

2) What’s the most you should be willing to pay for it?

3) What’s the most you would be willing to pay for a perfect test?

Update: Answer key after the jump.

These answers are much more long-winded than would be required for full credit.

Part I Answer as briefly as possible:

1. What policy question does the benefit-cost calculation allow you to answer that a cost-effectiveness analysis can’t answer?

What evaluative question does a cost-effectiveness analysis allow you to avoid that a cost-benefit analysis requires you to address?

Give an example (with made-up numbers) of a simple benefit-cost analysis and a simple cost-effectiveness analysis applied to the same problem.

Benefit-cost calculations can answer the question of how big a program’s budget should be, and the related question of whether money should be moved from one program to another.

Cost-effectiveness analysis doesn’t require assigning a monetary value to either the benefits or the costs of a program.

In digging the foundation of a building, if the choice is between using men with shovels who cost $10/hr. and can move 10 cubic yards of dirt per hour each, and doing the same job with a backhoe that, with its crew costs $100/hr and can move 200 cubic yards of dirt per hour, the backhoe is more cost-effective, since it moves dirt at 50 cents per cubic yard while the manual process costs a dollar a cubic yard.

But that doesn’t tell you how deep to dig the foundation. Say that digging the foundation a yard deeper would reduce maintenance costs by an amount whose present discounted value is $10,000, and that the building is 100 yards x 100 yards. (Assume the other construction costs would be identical.) Using the backhoe, we can move 10,000 cubic yards for $5,000. Thus the additional foundation depth passes the benefit-cost test; we should ask whether we can do still better by digging still deeper.

2. What inefficiency results if consumers are required to pay in order to use non-rival goods? How is it that consumers could be made worse off if producers of non-rival but potentially excludable goods are not allowed to make them actually excludable (that is, not allowed to deny access to the goods to those who do not pay)? Give an example of each.

If consumers are required to pay to use a non-rival good, those who value that good at more than zero but less tha the assigned price will abstain from using it. Their lost consumers’ surplus is an efficiency loss. Anyone who doesn’t listen to a piece of copyrighted recorded music he would otherwise listen to, or use a piece of copyrighted software he would otherwise use, because he doesn’t want to pay the download fee, provides an example. So does anyone who doesn’t buy an expensive patented medicine because of the price, who would be willing to pay the cost of production. (Each pill or bottle of pills is a rival good, but the knowledge embodied in the drug is non-rival. So the marginal cost is way below the average cost, which includes the research needed to develop the drug and get it approved.)

But if no one pays for music or software or drugs, then we should expect less music to be recorded, less software developed, and fewer new drugs invented. That will tend to make consumers worse off in the long run. Consumers of non-rival goods that cost something to develop in the first place face a public-goods problem, and charging for the use of those goods is one way of avoiding the free-rider problem.

3. What would you need to know to determine how much individual social capital (relational assets) a given person has?

You would need to know the list of people willing to do something to benefit that person, what costs they are prepared to incur to do so, and what resources (including their own individual social capital) they have available to them.

4. How does reputational externality create a collective-action problem? Give an example (actual or made-up) that illustrates the nature of the problem.

Insofar as the reputation of a group (Americans, trial lawyers, used-car dealers, Quakers, Rotarians, diamond merchants, men trying to pick up women at discos, ex-convicts, Marines, University of Maryland MPP graduates), matters to its members, that reputation is a public good for them; those who damage the group reputation and those who improve it may acquire different personal reputations, but those who deal with them but don’t know them personally will act according to their ideas of about group members generally. That creates a collective-action problem: the level of group-reputation-enhancing behavior optimal for any one selfish individual will be below the level that would be optimal considering the benefits to the entire group, and the personally optimal level of group-reputation-damaging behavior will be higher than the collectively optimal level. Groups differ in their capacity to get their members to approach the collective optimum; that capacity is one aspect of group (as opposed to individual) social capital.

During the Middle Ages, when the Jews of Europe were a recognizable and widely-disliked minority subject to official discrimination and occasional mob action, the rabbis decided that it was a violation of religious law for any Jew to act in a way that brought discredit on the Jews in the eyes of non-Jews. Formally, we could think of that as a way of internalizing the reputational externality, or, in Mancur Olson’s terms, creating private incentives for the provision of a public good.

5. Assume the expected value of the punishment for parking illegally is a $30 fine. If people made decisions about parking based on strictly selfish rational calculations and had diminishing marginal utility of income, would there be more illegal parking if:

a. Everyone who parks illegally is fined $30; or if

b. One illegal parker in 100 is chosen randomly and fined $3000?

Why is that so?

Name two theories about imperfect rationality that, if they explained actual behavior better than the rational-actor theory, would change your conclusion, and briefly explain why.

If the decisions were made purely rationally, a small chance of a big penalty would be a better deterrent than the certainty of a small penalty. Since the expected values of the penalties are the same, the question reduces to one of risk-preference. Anyone with diminishing marginal utility of income should be risk-averse, because the gain from a gamble adds less utility than the loss of the same amount subtracts.

However, if parkers behave as described in prospect theory, which means that they are behaviorally risk-loving in losses, then the smaller, more certain punishment may be more effective. The same is true if they suffer from optimistic biases; each may think his chance of being punished is less than 1 per 100.

6. What distinguishes a high discount rate from “temporal myopia,” as predicted for example by hyperbolic discounting? Give an example where you could tell which sort of behavior pattern some individual was displaying.

A high ( but geometric) discount rate leads to decisions that are consistent over time, while temporally myopic preferences such as those described by hyperbolic discounting lead to preference reversals: given a choice between a smaller reward sooner and a bigger reward later, the temporally myopic chooser may prefer the bigger-later reward at a distance in time, actually take the smaller-sooner reward if given the choice when that reward becomes available, and then regret the decision afterwards and even take steps to prevent himself from making similar bad choices in the future.

In Schelling’s example, if you office workers order lunch in advance, they will consume fewer calories than if they order at lunchtime, and they will continue to do so even after they discover at lunchtime that they really (temporarily) wish that they’d ordered the chocolate cake. That’s consistent with temporal myopia. Someone who just had such a high discount rate that the moment on the lips had greater value than the lifetime on the hips would go ahead and order dessert, even in advance.

7. How does risk-spreading trade off against incentive effects? Give an example.

The more people are compensated for their losses, the less cautious they need to be in avoiding them. Any insurance policy, or functional equivalent, therefore creates moral hazard as long as outcomes depend at least in part on behavior. Deductibles, co-payments, and experience rating all create incentives for caution, but at the expense of reducing the amount of risk-spreading. For example, insurance against workplace injury creates benefits by reducing risk, but it is notorious among physicians that injuries subject to worker’s compensation claims are much slower to heal than other injuries.

8. How does the theory of conspicuous consumption relate to the idea of market signaling? In what way is the result a market failure? Give an example.

There are great benefits to being thought wealthy. Any given expenditure will be less burdensome the greater the income of the person making it. So expenditure is a signal of wealth. The result may be that everyone spends more than he wants to on items that aren’t intrinsically worth their cost, but any individual who acts otherwise will suffer in reputation. In those circumstances, everyone would be better off if everyone spent less, but no individual can make himself better off by spending less personally: a typical collective-action or multi-player prisoner’s dilemma problem. If all the law students who spend $1500 on clothing for job interviews could make and enforce a bargain among themselves to spend no more than $1000, they might all be made better off.

Part II

You are the Director of Public Works in a town with a total annual budget of $3M. You need to build a temporary building to use as a Town Hall while the main Town Hall is reconstructed. Reconstruction will take three years. (Assume, implausibly, that there is no uncertainty about how long it will take.) Once the remodeling is done, the temporary building will be torn down.

A. You have two construction options. Plan A cost, ns $1M to build (including the cost of borrowing the money to build it) and $200k per year to operate. Plan B costs $1.1M to build (again, including financing costs) and $100 k per year to operate. They take the same amount of time to build, and are equally attractive and convenient.

What do you need to know in order to make the decision? What is the critical value of that variable?

You need to know the city’s discount rate. The critical value is the value of the discount rate at which the savings in maintenance just counterbalance the additional construction cost. Taking half of each year’s maintenance cost as being due at the beginning and half at the end, the answer is about 175%. So unless the city has to borrow from loansharks, it should spend the extra money on construction. If we treat the maintenance cost as being due at the beginning of the year, there’s no discount rate that equalizes the cost. If we treat it as being at the end of the year, the critical value is about 75%.

[Note: This question was sloppily constructed and required much more algebra than I intended. The alternative to solving the cubic equation is to make a guess at the critical value, compute the answer, and then adjust until it comes out close. ]

B. Now assume you’ve chosen Plan A over Plan B. Someone proposes Plan C instead. It has the same operating costs as Plan A, but the construction cost will be either $900,000 (if the job can be done without sinking pilings) or $1.1 million (if pilings are needed). There’s no way to tell for certain whether pilings are needed until construction has started, after which it would be too costly to switch back to Plan A.

Your geologist says it’s about an even chance whether pilings would be needed or not. Thinking in terms of the interests of the taxpayers, which would you choose, and why?

The expected value is the same. As long as the city can budget so that more important projects have priority over less important ones, it ought to be risk averse to a sum that’s nearly 3% of its budget (and a much larger proportion of its discretionary budget). So stick with Plan A.

C. (20 points) A firm of consulting geologists offers to do a preliminary soil analysis for $20,000. The analysis is not perfect. Its sensitivity (to the need for pilings) is 80%; its specificity is 75%. Based on the results of the test, you can buy actuarially fair insurance that will guarantee a fixed construction cost.

1) Is the test worth the proposed price.

2) What’s the most you should be willing to pay for it?

3) What’s the most you would be willing to pay for a perfect test?

Without the test, you will follow plan A. So we can set $1 million as the baseline. Going with Plan C and not needing pilings saves $100,000; going with Plan C and not needing pilings costs $100,000.

With a 50% prevalence, an 80% sensitivity, and a 75% specificity, the results of the test are:

40% true positives

10% false negatives

12.5% false positives

37.5% true negatives

So the test will show “positive” (i.e., predict that pilings are needed) 52.5% of the time. That reading will 76% accurate (40/52.5). To take even odds when your chance of winning is less than a quarter is silly. So you would go with plan A, getting neither a gain nor a loss from baseline (except for the cost of the test).

The test will show “negative” (no pilings needed) the other 47.5% of the time. That reading will be 79% accurate (37.5/47.5). Then the expected result of going with Plan C is

.79 (+100,000) + .21 (-100,000),

or a saving of $58,000, which you can lock in with the available insurance policy. (Since the expected cost of building is 900,000 plus 21% of $200,000, an actuarially fair policy to insure against the risk of needing to sink pilings would be $42,000, which saves you $58,000 compared to Plan A.)

But you only get that saving in case the test is negative, which happens 47.5% of the time. Thus the expected-value saving from running the test is 47.5% of $58,000, or $27,550. Since the cost of the test is $20,000 (no matter how it comes out), on average the test saves you $7,550 more than it costs. That’s the expected value of the test. If the city were risk-neutral for sums of that size, you should be willing to pay up to $27,550; but since it ought to be a little bit risk-averse, then you should be willing to pay a little bit less.

A perfect test would show negative 50% of the time, and guarantee you a savings of $100,000 each time it shows negative. So the expected value of perfect information is $50,000.