What Arne Duncan should learn from Eric Shinseki’s mistakes

Eric Shinseki falls victim to the cult of setting “stretch” numerical targets. Ed reformers please take note.

One of the following propositions is true:

1. Requiring subordinates to report numerical results will lead them to produce better results.

2. Requiring subordinates to report numerical results will lead them to report better results.

Eric Shinseki is now the former Secretary of Veterans Affairs because he, like many of his colleagues in the Obama Administration – especially at the Office of Management and Budget – didn’t understand the difference between those two propositions. The scandal wasn’t really his fault, but he still had to take the fall.

Apparently the systematic fudging of the waiting-list numbers was known to the Bush the Lesser administration and had started even earlier, but Shinseki was a strong advocate of numerical goal-setting, and in particular the strategy of setting “stretch” (i.e., impossible-to-satisfy) goals as a way of motivating extra effort. (One VA health-service provider, a sound progressive, told me back in 2010 that she was so frustrated at having to deal with idiotic goals imposed from DVA headquarters that planned to vote for any Republican against Obama in 2012.)

In fact, what “stretch” goals motivate is mostly deception.  If there’s no honest way to “make your numbers,” cheating seems like the only sensible strategy.

Understanding that insight doesn’t entail abandoning goal-setting and measurement. It does mean setting goals that make sense, and doing so in consultation with the people who have to meet them. And, most of all, it means creating a parallel system of audit so the folks at the top can discover that they’re being bamboozled by their subordinates other than by reading about it in the newspapers. In a sane world, this – rather that stupid games of “gotcha” about the price of muffins – would be one major role of the inspectors-general.

But even that won’t help if the folks at the very top have gambled their own careers on being able to report the satisfaction of unsatisfiable goals. The first step in knowing the truth is wanting to know the truth. As Machiavelli said and every con-man understands, whoever wants to deceive needs to find a victim who wants to be deceived. *

The ultimate blame for long waiting times at VA hospials falls, first on the war-lovers who made the decisions that produced a wave of wounded warriors, and secondly on the appropriators who didn’t provide the VA health system with budgets to match its burdens. If Sec. Shinseki had said to his subordinates, “These are the goals. What resources do we need to meet those goals?” and reported the answers as part of the appropriations process, the second part of that would have been obvious.

*Footnote What’s astounding is that the “education reformers,” including Arne Duncan, don’t seem to have learned the lesson, after years of what turned out to be fraudulent “miracles” in educating children from deprived backgrounds.

Measurement is essential, and I refuse to accept the ed-school and teachers-union bushwa about the ineffability of teaching. But measuring the wrong thing, or creating a measurement system that induces systematic cheating, is worse than not measuring at all. Measurement needs to grow out of the productive process rather than being imposed on it, it needs to inspire emulation rather than relying on fear, and it needs to foster rather than destroying the morale of the workforce.

Bill Bratton made the New York and Los Angeles police departments high-morale, results-driven organizations, but he didn’t do so by breaking the police unions, by denigrating his troops, by threatning to close underperforming precincts or cut the pay of underperforming cops, or by promising bonuses for cops who made their numbers. He understood that policing is a team sport. So is education. What American public management needs is less Frederick Taylor and more Edwards Deming.

Author: Mark Kleiman

Professor of Public Policy at the NYU Marron Institute for Urban Management and editor of the Journal of Drug Policy Analysis. Teaches about the methods of policy analysis about drug abuse control and crime control policy, working out the implications of two principles: that swift and certain sanctions don't have to be severe to be effective, and that well-designed threats usually don't have to be carried out. Books: Drugs and Drug Policy: What Everyone Needs to Know (with Jonathan Caulkins and Angela Hawken) When Brute Force Fails: How to Have Less Crime and Less Punishment (Princeton, 2009; named one of the "books of the year" by The Economist Against Excess: Drug Policy for Results (Basic, 1993) Marijuana: Costs of Abuse, Costs of Control (Greenwood, 1989) UCLA Homepage Curriculum Vitae Contact: Markarkleiman-at-gmail.com

14 thoughts on “What Arne Duncan should learn from Eric Shinseki’s mistakes”

  1. I think the problem is that these new reformers are often coming out of the corporate MBA world, where the numbers are everything towards climbing the ladder in the corporate office (or getting a bigger pay hike at the top). So they come in, deeply skeptical about the people working there that they have to manage, and slam down numbers goals in the name of accountability. Of course, they're not the same – the police department and schools don't give you the feedback in the form of going out of business if you're screw up, they just get worse and produce a whole bunch of disgruntled police officers, teachers, and students.

  2. The difference between Arne Duncan's and Eric Shinseki's positions is only this: there aren't many Democrats in Washington who are willing to sell off the VA to our kleptocratic elite, but there are plenty willing to sell off the public schools.

  3. regarding the footnote: could you tell me what the right thing to measure would be? all of the things you say in that paragraph sound good but what is it you think would be the appropriate thing to be measuring. related to that is how to incentivize improvements in what you choose for your metric. i've been teaching in texas for 19 years. we're the state that got suckered by h. ross perot 30 years ago into believeing that there was a crisis in our education system that could only be solved by throwing money at testing companies and then managed to take the show on the road when george w. became president. over the last 19 years i've watched the goalposts moved on an almost regular basis with each testing regime being said, in turn, to be little more than a literacy test and so replaced by a test that was said to be more "rigorous." basically every time texas teachers have figured out how to teach to a particular test and most schools are able to get most kids to pass the test the state declares the test is too easy and changes it. since it's illegal for teachers here to unionize i really don't know what this teacher-union bushwa thing is you speak of but at this point i'm desparately ready to reach a point where i can retire out of this treadmill.

    1. I don't think we know what to measure.

      In my view, the success of any educational endeavor is measured at the next level. Do you want to know how good my doctoral program is? Look at how much success its graduates have had. Do you want to know how good Clyde Tombaugh Elementary school is? Look at how its students do in middle school.

      The problem with this approach is twofold. First, it's hard. You must track students through the system. Students whose parents move often (as mine did) just complicate things. Do you attribute whatever success I've had to Pearl Harbor Elementary, Aliumanu Elementary, Ewa Beach Elementary (all on Oahu), or de Silva Elementary (in Hilo)?

      Second, there is the confounding problem. Most districts are organized to shuttle kids from several Elementaries to one Middle School and several Middles to one High School. When they get to Erehwon Central High School, the elementary school results are confounded with the middle schools.

      Finally, even after you've decided to measure something like the right population, what will you measure? Test scores? We've been down that road already.

      1. How about if we measure learning over knowledge and skills (use of the knowledge) over content. Ultimately, if a fourth graders comes in at the beginning of the year knowing fourth grade content, there is no learning going on that year. If the same fourth grader knows the content but can’t use it practically, the knowledge doesn’t serve the knower.

        Example: The Declaration of Independence was “signed” on July 4, 1776. (Knowledge a 4th grader might know or learn in 4th grade)

        The Declaration of Independence, signed on July 4, 1776 is a founding document (or precursor) to the Emancipation Proclamation and Gettysburg address, so likely influenced the Civil War and Lincoln’s efforts to abolish slavery. (Skill of developing context for understanding of history.)

  4. Bratton may have used (and may still use) the management tool Compstat to improve police performance, but his protege Garry McCarthy, head of the Chicago Police Department, seems to be using it to reduce crime the old-fashioned way. Chicago Magazine has run a couple of articles about the "phenomenal" reductions being posted for crime in Chicago. For a cynical look at the way Chicago crime statistics are being manipulated, you might look at the blog Second City Cop.

    1. There were plenty of reports at the time that Bratton’s CompStat implementation was being gamed, and since then the manipulation of number has led to at least a couple of scandals.

      Anecdotally speaking, numbers are great for figuring out where you need to apply additional resources, but only very rarely good for deciding whether to reward or punish. If you’re using numerical goals as your primary decisionmaking tool, you’re pretty much admitting you have either no ability or no resources to evaluate your staff.

  5. Goodhart's Law, formulated 1975. The most famous example is the systematic use of numerical targets in Soviet central planning, coupled to strong incentives for managers: under Stalin, survival; under his successors, money and status. Beyond a point, subordinates just make stuff up.

  6. And, most of all, it means creating a parallel system of audit so the folks at the top can discover that they’re being bamboozled by their subordinates other than by reading about it in the newspapers.

    This is indeed critical. So is not appointing managers who "don't want to hear negativity," as they put their refusal to hear bad news.

    You need people willing to report bad news, and in a position where they can do so fearlessly.

    Indeed, I suspect that problem contributed mightily to the disastrous ACA launch.

  7. ["Measurement is essential…[M]easuring the wrong thing, or creating a measurement system that induces systematic cheating, is worse than not measuring at all. Measurement needs to grow out of the productive process rather than being imposed on it, it needs to inspire emulation rather than relying on fear, and it needs to foster rather than destroying…morale…"

    This needs to be included in the instructions to any committee/task force/manager anywhere who is developing an assessment system…

    1. See almost any of the collected works of W. Edwards Deming. This paragraph could have come straight from W.E.D.

      It is unfortunate that the United States has still not figured out that Deming is more than control charts.

  8. I think the ed "reformers" made the first fundamental error, that of not wanting to know the truth. And the reason is, they thought they already knew it.

    And it conveniently turned out that "the problem" was defined by them to be unions made up mostly of older women, in an era of union-bashing. So, don't look for this issue to get fixed anytime soon. But, can we please have a challenger?

  9. Amen! I forget which book details Bratton's creative approach to crime reduction – partly the broken windows theory, which included repainting graffiti'd train cars overnight, so graffiti artists would not have the satisfaction of seeing their work "in action," busting turnstile jumpers so subway riders felt safer (and catching a lot of folks with outstanding warrants at the same time).

    The parallel audit system is smart. In the IT world, you can set up incentives to find holes in a system. You're essentially paying people to hack/try to break your system, and then using the results to patch the holes.

    Maybe the VA needs a version of the secret shopper.

Comments are closed.