Experimental findings on HOPE probation

In light of the impressive success of the Hawaii HOPE program, which brought attention to the application of Swift-Certain-Fair (SCF) principles in community corrections, and the spread of such programs nationally, the Bureau of Justice Assistance authorized a large-scale, four-site field trial (the Demonstration Field Experiment, or DFE) to examine how well such a program would do in other jurisdictions. Those results are now in – in the form of a paper by Pam Lattimore and her colleagues at the Research Triangle Institute (RTI)- and the findings are not especially favorable. It appears that HOPE probationers committed no fewer new crimes and spent on average somewhat more time confined in jail or compulsory residential treatment. The authors conclude that “HOPE/SCF seems unlikely to offer better outcomes and lower costs for broad classes of moderate-to-high–risk probationers.”

The publisher decided to make the Lattimore et al. paper freely available, but (over the protests of the editors) chose not to do so for the other papers in the same issue, including a number of commentaries on Lattimore et al.,  two of them sharply critical.

The short version of the critiques is that the conclusion drawn by Lattimore et. al goes far beyond the data. In particular, it conflates a specific program design (HOPE) with a general set of operating principles (Swift-Certain-Fair) and incorrectly concludes that since HOPE didn’t improve matters in four counties, SCF is unlikely to work anywhere.

Here’s the longer version:

(1) Many other studies of SCF programs show good results: not only the original Hawaii program but also 24/7 Sobriety in South Dakota, Swift and Certain in Washington State (a glowing evaluation of which appears in the same issue as the Lattimore et al. paper), and SWIFT in Texas. Pilot programs using SCF principles to reform the prison-discipline systems in Washington State, Ohio, and Pennsylvania all seem to be working well, though formal evaluations have yet to be published.

(2) The experimental set-up imposed a rigid formula – modeled after the Hawaii program – on the four DFE districts, whose  cultures, institutions, and circumstances bore very little resemblance to Hawaii’s.

(3) The model imposed by BJA used only jail as a sanction. Later implementations have had success in encouraging compliance relying mostly on non-jail sanctions, and making extensive use of rewards.

(4) The sanctions used (averaging more than 10 days per violation) were far more severe than those used in other, successful implementations. Hawaii HOPE now uses two days for most violations; 24/7 typically uses a single day. Experience suggests that the effectiveness of swift and certain sanctions is completely, or almost completely, independent of their severity. If the experimental programs had used less drastic sanctions to achieve the same violation rates , the days-behind-bars comparison – which largely drove the cost comparison – would have come out strongly in favor of SCF.

(5) An excessively long list of rules to follow (“conditions of probation”) is a well-known problem in community corrections. No one can remember sixty rules, let alone obey all of them. Since swift-certain-fair programs insist on imposing a sanction for every detected violation, an excessively long list of conditions is a formula for failure. That suggests that one element of SCF program design should be a radical pruning of the conditions list, imposing on each probationer only those rules required for public safety and rehabilitation.  (Ron Corbett, who used to run Massachusetts Probation, calls this “zero-based conditioning.”) But zero-based conditioning was not practiced at the DFE sites; that committed those probation departments to rigidly enforce rules that shouldn’t have been there in the first place. In particular, some of the experimental jurisdictions jailed people for not paying probation fees, which was never part of the Hawaii model and obviously violates the principle of fairness.

(6) Imposing a rigid program design prevented the process of consultation with stakeholders which is crucial to the success of any program. Key elements of the Hawaii program were invented, not by Judge Steven Alm, but by the Public Defender (the warning hearing) and the probation department (a streamlined process for reporting violations to the court).

(7) Rigid program design also meant that the programs under study couldn’t be modified in the face of operational problems, but had to be carried forward according to the experimental protocol. Successful programs adapt over time.

(8) Every SCF program takes time to build credibility with those being supervised, so best practice is to roll a program out and shake it down operationally first, and start evaluation later. The Lattimore et al. evaluation started on Day 1. As expected, the first batch of entrants had much worse outcomes than subsequent batches; the bad result reported was the average of the bad early results and the better, later results. But the later results are more representative of long-run performance than the rocky initial results.

(9) Community corrections, like the rest of the criminal justice system, is plagued by racial and ethnic disparities in treatment. In Hawaii, analysis of the data showed that Native Hawaiians and Samoans on routine (non-SCF) probation were more likely to face punishment for any given violation than was the case for the rest of the population. The same was true of African-Americans in Washington State. In both cases, the reduction in discretion as a result of the SCF process – because probation officers are committed to reporting violations to the court and the court is committed to consistent rather than sporadic responses – translated into a virtual disappearance of racial and ethnic disparity. Since reducing disparity was not among the announced goals of the program, the RTI team did not examine whether similar results were achieved in the DFE.

(10) One dramatic success of SCF programs – notably Hawaii HOPE and 24/7 – is in reducing drug use. The Hawaii program succeeded in getting about 80% of its heavily methamphetamine-using probationers to stop using, and the benefits lasted even after the subjects were released from supervision. However the Lattimore et al. evaluation did not even report on drug use as an outcome, and therefore couldn’t consider the benefits of reduced drug use in its benefit-cost analysis.

If the question addressed was “Is HOPE a magical program-with-a-manual that will succeed everywhere if mindlessly replicated?” Lattimore et al. show that the answer is “No, it isn’t.” But that’s not a surprise to the people who have been out there doing this work; if asked that question, most would reply, “Of course not!”

The conclusion that seems to flow from considering all the available data, rather than the DFE analysis in isolation, is that systems of swift, certain, and fair incentives (rewards as well as sanctions) can and do succeed when implemented in consultation with practitioners and in a form consistent with local conditions. One key to success is minimizing the use of jail as a sanction. Programs mismatched to local conditions, or which violate the principles of fairness and procedural justice (including demonstrated goodwill toward participants) perform less well.

SCF is an operating concept for corrections, not a specific program. How to fit that concept to local conditions is always a complex problem. But unless and until someone produces a logical argument in support of “slow, random, and arbitrary,” there is no reason to ignore the growing evidence that swift, certain, and fair tends to lead to better outcomes.

Author: Mark Kleiman

Professor of Public Policy at the NYU Marron Institute for Urban Management and editor of the Journal of Drug Policy Analysis. Teaches about the methods of policy analysis about drug abuse control and crime control policy, working out the implications of two principles: that swift and certain sanctions don't have to be severe to be effective, and that well-designed threats usually don't have to be carried out. Books: Drugs and Drug Policy: What Everyone Needs to Know (with Jonathan Caulkins and Angela Hawken) When Brute Force Fails: How to Have Less Crime and Less Punishment (Princeton, 2009; named one of the "books of the year" by The Economist Against Excess: Drug Policy for Results (Basic, 1993) Marijuana: Costs of Abuse, Costs of Control (Greenwood, 1989) UCLA Homepage Curriculum Vitae Contact: Markarkleiman-at-gmail.com

6 thoughts on “Experimental findings on HOPE probation”

  1. Knowing nothing more about the study than what Mark has written, it seems to have been a badly flawed design. I've been hoping that SCF, and coupled with greatly improved monitoring bracelets, would offer some kind of door out of the incarceration tunnel, so it's distressing to think it might get derailed by a poorly designed study. This leads to the (boring, drab, banal) conclusion that there should be more thoughtful care in designing studies, and maybe by boosting the number of skeptical policy weenies working for grant agencies.

    1. Yes. It seems distressingly common to have studies that were so small or badly designed that they could not possibly lead to strong and useful conclusions. Perhaps there needs to be more peer review at the design stage.

  2. This is also why the usual complaints about cherry-picking of measures in research has to be taken with a grain of salt. Sometimes if you just gather enough data and apply enough tests you'll get a positive result by chance, but sometimes if you just gather enough data and take enough slices through it you'll get something significant that's not by chance.

  3. HOPE probation is a self promoted program that relies on illegalities from one judge in Hawaii bullying the legal system
    with the blessing of criminal attorneys and the help of the State Attorney General office that is preparing the data.
    The statistics from these biased data were exploited by one(!) academic group which is not even composed of criminologists
    but faculties from a very mediocre school of Public Policy. "Judge" Alm is asking probationers to take responsability for
    their acts while he told a criminal lawyer "it is illegal but if you do not complain about it, I will do it."
    (recording at http://youtu.be/7JFcQ6jrj2w)

    As the last example of HOPE series of victims, in April 2016, 16 year old Wesley Lasalo Ulugalu-Sula was stabbed to
    death in Honolulu, Hawaii by a HOPE probationer that was illegally kept on probation by Judge Alm. How many more victims
    to suit statistics?

    And the cherry on the cake: the author of this article is from that only mediocre academic group that promoted HOPE
    (or themselves?) from the beginning; again he is is no criminologist but a professor in public policy!

    Issues and crimes about HOPE probation have been reported on the Facebook page: https://www.facebook.com/pages/HOPE-Probation-the

  4. There is one very true comment in this word diarrhea: "circumstances bore very little resemblance to Hawaii’s"
    Indeed there is no other State whose judicial system is as corrupted as that of Hawaii with a filthy State Attorney General office biasing data to allow a crooked judge to gain federal funding…

  5. As the last example of HOPE series of victims, in April 2016, 16 year old Wesley Lasalo Ulugalu-Sula was stabbed to
    death in Honolulu, Hawaii by a HOPE probationer that was illegally kept on probation by Judge Alm.
    Based on the records available, Bringas was put on HOPE probation on 10/20/2009 and his probation was revoked on 05/13/2016
    following the murder. During the period 10/20/2009-05/13/2016 the maximum time he should have spent in jail was 1 year and 1/2
    (according to Hawaii Revised Statute (HRS) 706-624-(2)-(a)). He certainly got jail time in 2009 when he was found guilty of the
    class-B felony (the record does not indicate how much) and got 17(!) new charges before the murder. In particular, in 2012 and
    2015 he picked each time one year in jail: on the same term of probation this is illegal! (without commenting on being kept on
    probation despite 17 new charges!)
    HOPE probation cases like that were recorded as one probation revocation by the Hawaii State Attorney General Office and the
    reduction of the number of probation revocations compared to standard probation given as a positive statistics in HOPE studies:
    how many murders were committed under HOPE to suit statistics?
    It can be noted as well that the State of Hawaii Commission on Judicial Conduct was informed of Alm's probation revocation
    illegalities in 2014 and could have prevented that 16 year old murder by simply following on it at the time.

Comments are closed.