Retraction: NCLB isn’t mathematically impossible

My hearty thanks to Andrew Rotheram, the Eduwonk, for correcting a misimpression under which I had been laboring.

It’s well known that the No Child Left Behind law requires that by 2012 every school be operating up to standard in its educational peformance with respect to every population subgroup. I had understood that as meaning that every school be up to average, which is of course absurd: outside Lake Wobegone, there’s no such thing as a distribution in which every element is at or above the mean.

Rotheram points points out in an email that the law permits the use of “criterion-referenced” tests rather than “norm-referenced tests.” Since the standards on “criterion-referenced” tests don’t change as the scores change, it’s not impossible for every school to score “above average” sometime in the future if the criteria for what counts as “average” are set now and not modified as school performance improves.

So I retract my claim that NCLB mandates a mathematical impossibility. (It still seems pretty silly to say that all schoools have to be above average, even if you don’t mean it, but let that pass.)

My two other criticisms, one technical and one fundamental, remain.

The technical objection is that statistical “noise” will lead to smaller schools’ thrashing back and forth between being rewarded for extraordinarily good performance and being punished for substandard performance unless the scores are adjusted for their standard errors.

The fundamental problem is that the insistence on testing every student pushes school systems toward machine-scoreable tests and thus away from tests that measure the full range of outcomes we’re interested in, leading to curricular pressure in the wrong direction, to teaching to the test, and to outright cheating. It still seems to me that the principles of statistical quality assurance would dictate administering much more expensive and informative tests on a sample of students rather than relying on census testing.

Author: Mark Kleiman

Professor of Public Policy at the NYU Marron Institute for Urban Management and editor of the Journal of Drug Policy Analysis. Teaches about the methods of policy analysis about drug abuse control and crime control policy, working out the implications of two principles: that swift and certain sanctions don't have to be severe to be effective, and that well-designed threats usually don't have to be carried out. Books: Drugs and Drug Policy: What Everyone Needs to Know (with Jonathan Caulkins and Angela Hawken) When Brute Force Fails: How to Have Less Crime and Less Punishment (Princeton, 2009; named one of the "books of the year" by The Economist Against Excess: Drug Policy for Results (Basic, 1993) Marijuana: Costs of Abuse, Costs of Control (Greenwood, 1989) UCLA Homepage Curriculum Vitae Contact:

One thought on “Retraction: NCLB isn’t mathematically impossible”

  1. The Strange Case of NCLB Standards

    Having been straightened out by Andrew Rotherham (and, passim outing him as the author of Eduwonk) Mark Kleiman reconsiders No Child Left Behind, offering some cogent critiques. Cogent critiques, though, are slightly besides the point because they miss…

Comments are closed.