Deming, thou should’st be living at this hour

Sharon Otterman reviews the failing, flailing efforts of New York City to improve teaching the easy way.  The story, and the management it describes, is all about trying to get quality by incentives and personnel, in particular, sorting teachers into good, bad, and indifferent bins by observing student test score changes over a year.  Presumably, those in the good bin get more pay, or something, and we fire the bad ones. The implicit model is that teaching is an irremediable trait among bad teachers, and purely a matter of incentives and motivation among all the others, and a positional arms race is just the ticket to make teachers really want to do a good job, right?

There’s so much wrong with this it’s hard to know where to start, even though test score increases is among the useful information that should be collected and analyzed, mostly to direct attention to teacher practice that seems to work and would reward analysis and discussion. But attributing outcomes to the teacher is nuts, as Deming demonstrated years ago.  First, the instrument is very noisy; the article begins with some anecdotes of teachers omitted from the system, teachers scored in years when they didn’t teach, and teachers given the wrong scores.  Second, stuff happens; every teacher knows that a class takes on a personality early in the year on the basis of unobserved or random events, or just the luck of the draw in student selection, that seems to be refractory to what a teacher does.  Some years have lots of snow days, some years there’s a shooting in the school, and on and on.  So the teacher effect is going to pick up correlations, spurious and real, with all the variables that are not observed.

Finally other stuff happens: Deming – brilliant, tough-minded, and humane –  demonstrated that if you reward individual workers for performance, you are going to be rewarding random variation a lot of the time, with poisonous effects.  Right away, when the top salesman among twenty gets a trip to Hawaii with his wife, the response of the other nineteen is not to emulate him (and how could they, if they don’t see what he does, which is the case for teachers in spades), but to be pissed off and jealous, which is, like, really great for collaborative enterprise.  Next year, regression toward the mean sets in and he is only number five, or ten, so he looks like a slacker, coasting on his laurels. Even his wife starts giving him the fisheye; don’t be surprised if his lunch martini count starts to go up.

It is a universal, desperate, desire of lazy or badly trained managers to find a mechanistic device you can wind up like a clockwork, loose upon the organization, and go play golf. Like testing and firing to get people to do good work.  Please, Lord, show me the way to manage without any actual heavy lifting!  But many desires, no matter how desperately we cleave to them, are not fated to be fulfilled, and this is one.  Teaching, like any complex production process, will get better when teachers watch each other work and talk about what they are doing, why, and how it works; what to watch is usefully indicated by statistical QA methods.  Period.  It was true in Smith’s pin factory, it was true in the opera staging master class at the New England Conservatory I sat in on back in the day, it is true in Toyota and, finally, GM factories, and it’s true in schools.  Deming was right, but as long as we’re afraid to admit it (one of his 14 principles is, ironically, “Drive out fear”) we will continue to leave value on the table, absurdly and tragically.