Experimental findings on HOPE probation

In light of the impressive success of the Hawaii HOPE program, which brought attention to the application of Swift-Certain-Fair (SCF) principles in community corrections, and the spread of such programs nationally, the Bureau of Justice Assistance authorized a large-scale, four-site field trial (the Demonstration Field Experiment, or DFE) to examine how well such a program would do in other jurisdictions. Those results are now in – in the form of a paper by Pam Lattimore and her colleagues at the Research Triangle Institute (RTI)- and the findings are not especially favorable. It appears that HOPE probationers committed no fewer new crimes and spent on average somewhat more time confined in jail or compulsory residential treatment. The authors conclude that “HOPE/SCF seems unlikely to offer better outcomes and lower costs for broad classes of moderate-to-high–risk probationers.”

The publisher decided to make the Lattimore et al. paper freely available, but (over the protests of the editors) chose not to do so for the other papers in the same issue, including a number of commentaries on Lattimore et al.,  two of them sharply critical.

The short version of the critiques is that the conclusion drawn by Lattimore et. al goes far beyond the data. In particular, it conflates a specific program design (HOPE) with a general set of operating principles (Swift-Certain-Fair) and incorrectly concludes that since HOPE didn’t improve matters in four counties, SCF is unlikely to work anywhere.

Here’s the longer version:

(1) Many other studies of SCF programs show good results: not only the original Hawaii program but also 24/7 Sobriety in South Dakota, Swift and Certain in Washington State (a glowing evaluation of which appears in the same issue as the Lattimore et al. paper), and SWIFT in Texas. Pilot programs using SCF principles to reform the prison-discipline systems in Washington State, Ohio, and Pennsylvania all seem to be working well, though formal evaluations have yet to be published.

(2) The experimental set-up imposed a rigid formula – modeled after the Hawaii program – on the four DFE districts, whose  cultures, institutions, and circumstances bore very little resemblance to Hawaii’s.

(3) The model imposed by BJA used only jail as a sanction. Later implementations have had success in encouraging compliance relying mostly on non-jail sanctions, and making extensive use of rewards.

(4) The sanctions used (averaging more than 10 days per violation) were far more severe than those used in other, successful implementations. Hawaii HOPE now uses two days for most violations; 24/7 typically uses a single day. Experience suggests that the effectiveness of swift and certain sanctions is completely, or almost completely, independent of their severity. If the experimental programs had used less drastic sanctions to achieve the same violation rates , the days-behind-bars comparison – which largely drove the cost comparison – would have come out strongly in favor of SCF.

(5) An excessively long list of rules to follow (“conditions of probation”) is a well-known problem in community corrections. No one can remember sixty rules, let alone obey all of them. Since swift-certain-fair programs insist on imposing a sanction for every detected violation, an excessively long list of conditions is a formula for failure. That suggests that one element of SCF program design should be a radical pruning of the conditions list, imposing on each probationer only those rules required for public safety and rehabilitation.  (Ron Corbett, who used to run Massachusetts Probation, calls this “zero-based conditioning.”) But zero-based conditioning was not practiced at the DFE sites; that committed those probation departments to rigidly enforce rules that shouldn’t have been there in the first place. In particular, some of the experimental jurisdictions jailed people for not paying probation fees, which was never part of the Hawaii model and obviously violates the principle of fairness.

(6) Imposing a rigid program design prevented the process of consultation with stakeholders which is crucial to the success of any program. Key elements of the Hawaii program were invented, not by Judge Steven Alm, but by the Public Defender (the warning hearing) and the probation department (a streamlined process for reporting violations to the court).

(7) Rigid program design also meant that the programs under study couldn’t be modified in the face of operational problems, but had to be carried forward according to the experimental protocol. Successful programs adapt over time.

(8) Every SCF program takes time to build credibility with those being supervised, so best practice is to roll a program out and shake it down operationally first, and start evaluation later. The Lattimore et al. evaluation started on Day 1. As expected, the first batch of entrants had much worse outcomes than subsequent batches; the bad result reported was the average of the bad early results and the better, later results. But the later results are more representative of long-run performance than the rocky initial results.

(9) Community corrections, like the rest of the criminal justice system, is plagued by racial and ethnic disparities in treatment. In Hawaii, analysis of the data showed that Native Hawaiians and Samoans on routine (non-SCF) probation were more likely to face punishment for any given violation than was the case for the rest of the population. The same was true of African-Americans in Washington State. In both cases, the reduction in discretion as a result of the SCF process – because probation officers are committed to reporting violations to the court and the court is committed to consistent rather than sporadic responses – translated into a virtual disappearance of racial and ethnic disparity. Since reducing disparity was not among the announced goals of the program, the RTI team did not examine whether similar results were achieved in the DFE.

(10) One dramatic success of SCF programs – notably Hawaii HOPE and 24/7 – is in reducing drug use. The Hawaii program succeeded in getting about 80% of its heavily methamphetamine-using probationers to stop using, and the benefits lasted even after the subjects were released from supervision. However the Lattimore et al. evaluation did not even report on drug use as an outcome, and therefore couldn’t consider the benefits of reduced drug use in its benefit-cost analysis.

If the question addressed was “Is HOPE a magical program-with-a-manual that will succeed everywhere if mindlessly replicated?” Lattimore et al. show that the answer is “No, it isn’t.” But that’s not a surprise to the people who have been out there doing this work; if asked that question, most would reply, “Of course not!”

The conclusion that seems to flow from considering all the available data, rather than the DFE analysis in isolation, is that systems of swift, certain, and fair incentives (rewards as well as sanctions) can and do succeed when implemented in consultation with practitioners and in a form consistent with local conditions. One key to success is minimizing the use of jail as a sanction. Programs mismatched to local conditions, or which violate the principles of fairness and procedural justice (including demonstrated goodwill toward participants) perform less well.

SCF is an operating concept for corrections, not a specific program. How to fit that concept to local conditions is always a complex problem. But unless and until someone produces a logical argument in support of “slow, random, and arbitrary,” there is no reason to ignore the growing evidence that swift, certain, and fair tends to lead to better outcomes.

Mass incarceration is a hard problem, not an easy one.

Freeing the innocent and harmless won’t solve over-incarceration.
How do you control the dangerously guilty?

Getting out of the mass-incarceration trap is hard. We currently have five times our historical incarceration rate and seven times the incarceration rate of most of Western Europe. We also have a frighteningly high rate of homicide and other violent crime, albeit only about half as high as it was twenty years ago.

To get back to our historical level, which would be a high level compared to the rest of the civilized world, we need to reduce the headcount behind bars by 80%. Pick five prisoners, and you need to let four of them out.

And no, you can’t do that by releasing innocent, harmless people. You need to release some seriously guilty and dangerous people. Which means you need to figure out a way to make them less dangerous that doesn’t involve putting them under lock and key.

Megan McArdle gets this right.

 

Concerning replication

Heraclitus said that no one can step twice into the same river. By the same token, it isn’t really possible to do the same program in two different places.

I’m in London for meetings on criminal justice organized by the Centre for Justice Innovation (the UK arm of the Center for Court Innovation) and Policy Exchange. A small meeting that ran for two hours today tried to figure out whether the coerced-sobriety approach of the South Dakota 24/7 project (the alcohol version of HOPE) could be used in Scotland, with social problems, customs, and institutional arrangements quite unlike South Dakota’s. (In particular, Scotland has no such thing as a two-day term of confinement.)

The answer seems to be that you could try to do something based on similar principles and with similar aims, but you couldn’t really do the program in its trademark style, and even if you’re convinced 24/7 works on drunk drivers in Sioux Falls you can’t be sure that the alternative version would work on drunken wife-beaters in Glasgow.

That suggests a more general reflection. Heraclitus said that no one can step twice into the same river. By the same token, it isn’t really possible to do the same program in two different places.

Coincidence

The Drug Policy Alliance has such great regard for the truth that its reports dole out the precious substance with a sparing hand.

This morning, I got an email from a friend who isn’t involved in the War About Drugs but knows a great deal both about drug abuse and about my views on the topic. Like me, he is neither a hard-core warrior nor a flat-out legalizer. He had just received a fund-raising pitch for the Drug Policy Alliance, signed by George Soros. (I should note that Soros is one of my heroes; I can’t think of an individual in our lifetimes whose philanthropy – especially in the former Soviet Bloc – has created more value for the world.)

My friend wrote, “It looks good. Is it?”

I wrote back “In a word, no. DPA is the main drug-legalization group. They’re not very forthcoming about their actual aims.”

Of course, that’s the point; DPA sounds much more reasonable on paper than it is in practice. You might not guess that its announced goal of “drug policies based on science, compassion, health and human rights” means in practice opposition to any effective form of drug abuse control.

Then, when I got to work, I found in my mail a slick (in both senses) report on drug courts from DPA. It’s unsigned, which I would say reflected prudence on the part of the author or authors; it is not the sort of document anyone with any scholarly self-respect would want his name attached to.

Now, I’m not a fan of drug courts as they currently exist – they enroll too many low-level offenders without severe drug problems who would have done better with a good leaving-alone – so many of the report’s criticisms of drug courts seem on target to me. But the level of sleaze in the argumentation is truly breathtaking.

Continue reading “Coincidence”