Jim Manzi points out that social-scientific knowledge is hard to come by: regression analysis often isn’t supported by controlled trials, and that even controlled-trial results require replication. From this he concludes that:

We should be very skeptical of claims for the effectiveness of new, counterintuitive programs and policies, and we should be reluctant to trump the trial-and-error process of social evolution in matters of economics or social policy … we need to keep stumbling forward with trial-and-error learning as best we can.

I’m sorry, but this is incoherent. What is this magical “trial-and-error process” that does what scientific inquiry can’t do? On what basis are we to determine whether a given trial led to successful or unsuccessful results? Uncontrolled before-and-after analysis, with its vulnerability to regression toward the mean? And where is the mystical “social evolution” that somehow leads fit policies to survive while killing off the unfit?

Without any social-scientific basis at all (unless you count Gary Becker’s speculations) we managed to expand incarceration by 500 percent between 1975 and the present. Is that fact – the resultant of a complicated interplay of political, bureaucratic, and professional forces – to be accepted as evidence that mass incarceration is a good policy, and the “counter-intuitive” finding that, past a given point, expanding incarceration tends, on balance, to increase crime be ignored because it’s merely social science? Should the widespread belief, implemented in policy, that only formal treatment cures substance abuse cause us to ignore the evidence to the contrary provided by both naturalistic studies and the finding of the HOPE randomized controlled trial that consistent sanctions can reliably extinguish drug-using behavior even among chronic criminally-active substance abusers?

For some reason he doesn’t specify, Manzi regards negative trial results as dispositive evidence that social innovators are silly people who don’t understand “causal density.” So he accepts – as well he should – the “counter-intuitive” result that juvenile boot camps were a bad idea. But why are those negative results so much more impressive than the finding that raising offenders’ reading scores tends to reduce their future criminality?

Surely Manzi is right to call for metholological humility and catholicism; social knowledge does not begin and end with regressions and controlled trials. But the notion that prejudices embedded in policies reflect some sort of evolutionary result, and therefore deserve our respect when they conflict with the results of careful study, really can’t be taken seriously.