Disappointing results on nurse home visits as crime prevention

At age 19, boys whose mothers received nurse home visits as part of a randomized controlled trial of the Nurse-Family Partnership program were no less criminally active than boys in the control group. Since crime is overwhelmingly a male phenomenon, the finding in the same study that NFP greatly reduced criminality among girls (4% arrested by age 19 v. 20%) isn’t much consolation.

The lack of impact on crime among males is even more striking given the dramatic improvements on other outcome measures. Those non-crime benefits more than cover the costs of the program. But those of us who thought that nurse home visits were going to turn out to be a hugely effective crime-control measure will have to go back to the drawing board.

Author: Mark Kleiman

Professor of Public Policy at the NYU Marron Institute for Urban Management and editor of the Journal of Drug Policy Analysis. Teaches about the methods of policy analysis about drug abuse control and crime control policy, working out the implications of two principles: that swift and certain sanctions don't have to be severe to be effective, and that well-designed threats usually don't have to be carried out. Books: Drugs and Drug Policy: What Everyone Needs to Know (with Jonathan Caulkins and Angela Hawken) When Brute Force Fails: How to Have Less Crime and Less Punishment (Princeton, 2009; named one of the "books of the year" by The Economist Against Excess: Drug Policy for Results (Basic, 1993) Marijuana: Costs of Abuse, Costs of Control (Greenwood, 1989) UCLA Homepage Curriculum Vitae Contact: Markarkleiman-at-gmail.com

7 thoughts on “Disappointing results on nurse home visits as crime prevention”

  1. Good grief — am I reading these stats right? Did you say 20% of girls in the control group had been arrested by age 19 but that this number is so small, relatively, that an 80% reduction — changing the lives of one girl in six — is no big deal? How high do the arrest rates for boys have to be to make that comparison work out — 97%?

  2. Yes, the reduction in crime among girls was impressive. But since boys commit the overwhelming bulk of all crimes, and especially violent crimes cutting crime among girls does very little to cut crime overall. Nurse home visits are well cost-justified without any crime control benefit. But if you were hoping, as I was, that you could substitute NHV for law enforcement, these numbers don't look good.

  3. There are 310 children in the study, 140 of whom were in the control group (73 female, 67 male); 79 received nurse visits when their mothers were pregnant (genders unspecified); and 91 were visited as infants, as well as when their mothers were pregnant (44 female, 47 male).

    So, what we know: 4% of the females in the doubly treated group had a conviction at age 19, versus 20% in the control group. (Mark, your post is in error — this is conviction rate, not arrest rate). That's 2/44 versus 15/73. I don't know who this control group of women was, but that looks like a statistical anomaly to me.

    Meanwhile, there were no notable differences between the 67 control group males and 47 treated males. But how could there be? These samples are ridiculously small, considering that even in high risk populations so few are convicted by age 19. A test like this has no power.

    Let's do a thought experiment. Suppose that NFP is ridiculously, shockingly effective and reduces crime by 1/5. If the control group had a "true" probability of 25% (the statement doesn't give us the numbers in the sample), and the treated group 20%, then on average we'd see 17/67 control groupies get convicted, and 9/47 treated guys. Mark's favorite online significance calculator tells us that this corresponds to a z-score of .4. About 35% of the time, you'd see the conviction rate of the treated group be higher than the conviction rate of the control group. (I could be bungling the numbers, but the idea is right).

    I think Mark's comment here is telling:

    The lack of impact on crime among males is even more striking given the dramatic improvements on other outcome measures.

    Ah, they're looking at a lot of outcome measures. Now it makes sense. You look at enough variables, and some of them are going to be significant. In this case, it happens to be criminality of girls, with a point estimate of reducing crime by 80%! And medical researchers write a separate article for every significant estimate from every study, so we end up with this.

    As I said above, I'm not saying that NFP isn't effective; rather, when you're looking at low probability events from an already tiny sample, there's no way of telling signal from noise even when the signal is strong.

    http://www.dimensionresearch.com/resources/calcul

  4. While I applaud the efforts of these researchers, I agree with Alex: the sample size is just too small to draw many conclusions.

    A few months back I read an article claiming that aerobic exercise curbs hunger pangs more effectively than strength training. After spending some of that day's juice reading this I found that the author wrote it based on an informal survey of 11 acquaintances. That was worse than a waste of time. One should review the study before reading the commentary I suppose.

    Mark, if I'm not mistaken, analysis of this sort of policy is your forte. Today's GOP would fight bitterly against the public library system, post office, etc if we were to try to create them now (whereas the founding fathers would doubtless support cheap broadband). I wonder how optimistic you are about your set actually having a voice now.

  5. When I read the summary last night, I thought that the sample size seemed quite small, too. Looking at the article itself (http://archpedi.ama-assn.org/cgi/reprint/164/1/9), the problem is even more obvious. For instance, on page 13, the authors tell us that the number of boys born from "high risk mothers" (mothers who were both low SES and unmarried) is 18, and in the control group n=23.

    Table 4, which outlines the overall affects of treatment (that is, lumping together boys and girls and including every "risk" category) may tell us something about their sample size, too. On educational attainment, teen pregnancy, likelihood to use Medicaid, or economic productivity, the study can't tell whether treatment makes any outcome more or less likely. The only result with a confidence interval that doesn't drop below 1 in terms of relative risk is for mothers who received pre-natal home visits but nothing after the child was born. There, the treatment children were somewhere between 14% and 705% MORE likely to have received welfare than the control children. That result is certainly possible, but it seems pretty unlikely.

    It's obviously difficult to get large sample sizes for 19-year longitudinal studies, but an n=18 for "high risk" boys is pretty damned low. To my mind, the only really credible numbers with any decent sample sizes are the cumulative effects (across all of the subjects) shown in Table 3. Even there, however, the only sound results (where the relative risk CI doesn't cross 1.0) are for lifetime arrests and convictions of children who kept getting home visits until the age of 2.

    I realize that part of my resistance to the results is obviously guided by the fact that I, like Mark, want nurse home visits to work in terms of crime reduction. There are a lot of reasons to think that the visits actually could work. If the evidence begins to show that I'm wrong, I'll be disappointed, but probably not dismayed to the point of cognitive dissonance. Standing alone, this study doesn't do much to convince me that the interventions don't work. Even its "encouraging" results don't inspire a whole lot of confidence in me. All it really tells me is that we need better data.

  6. This stuff is so tricky. There's just so many variables involved. NHVs are only one factor out of many. From what I've read, you can get good numbers on very specific outcomes a few years in. But to try and control things on longitudinal distances this large seems incredibly dicey. What is happening in these kids lives – in school, in families, etc., etc.?

    Harlem Children's Zone is getting huge graduation rates but with a plethora of services. Yet picking out what is operating where on a general level seems way beyond reasonable scope.

    I think if you can get solid data on specific efficacies then why worry about trying to extrapolate out so far – its bound to muddle the original gains. Let's take this stuff one step at a time and drill into what is working where and go from there.

  7. 1. Since the nurse home visits pay off anyway, it's worthwhile persevering and tweaking the interventions to try to be more effective with boys. Perhaps when gender differences really start to emerge (age 3 or 4, not stopping at 2).

    2. The control group isn't the general population: it's the mothers and children given medical screening and treatment in the same program, but not the nurse visits. One would expect the control program to offer some benefits on a variety of fronts, so masking the benefits of the nurse visits by themselves. For example, both could pick up on signs of abuse, anger issues, or malnutrition.

    3. Why the gender difference? The study doesn't (at a quick glance) say whether the mothers were single, but it's a fair bet that most were. The program seems to have succeeded in making the mothers better at the role: maybe more skilled, maybe just more confident and less worried. Possibly good mothering rubs off more on teenage girls than teenage boys, for whom the problem is the absent father, replaced by the peer group. This can't be fixed by this type of intervention. The alternative explanation that the nurses provided gender-stereotyped advice about infants that turned out useless for boys ("Don't give him / do give him toy guns!") doee not make sense.

    4. One of the strong results for girls was less teenage childbearing (11% vs. 30% in the control group). Surely this is a large long-term crime reduction bonus in the next generation?

Comments are closed.