My hearty thanks to Andrew Rotheram, the Eduwonk, for correcting a misimpression under which I had been laboring.
It’s well known that the No Child Left Behind law requires that by 2012 every school be operating up to standard in its educational peformance with respect to every population subgroup. I had understood that as meaning that every school be up to average, which is of course absurd: outside Lake Wobegone, there’s no such thing as a distribution in which every element is at or above the mean.
Rotheram points points out in an email that the law permits the use of “criterion-referenced” tests rather than “norm-referenced tests.” Since the standards on “criterion-referenced” tests don’t change as the scores change, it’s not impossible for every school to score “above average” sometime in the future if the criteria for what counts as “average” are set now and not modified as school performance improves.
So I retract my claim that NCLB mandates a mathematical impossibility. (It still seems pretty silly to say that all schoools have to be above average, even if you don’t mean it, but let that pass.)
My two other criticisms, one technical and one fundamental, remain.
The technical objection is that statistical “noise” will lead to smaller schools’ thrashing back and forth between being rewarded for extraordinarily good performance and being punished for substandard performance unless the scores are adjusted for their standard errors.
The fundamental problem is that the insistence on testing every student pushes school systems toward machine-scoreable tests and thus away from tests that measure the full range of outcomes we’re interested in, leading to curricular pressure in the wrong direction, to teaching to the test, and to outright cheating. It still seems to me that the principles of statistical quality assurance would dictate administering much more expensive and informative tests on a sample of students rather than relying on census testing.