The Hopelessness of Epidemiologic Understanding

At an alcohol conference in Ireland, a clinician colleague commented from the lectern that the UK safe drinking guidelines were “Puritanical” because “Me and my mates like to go out and have 4 or 5 pints on Fridays — how could that be hazardous?”. A fairly similar comment was related to me by a reporter recently, more along the lines of asking why public health guideline writers are such killjoys when it comes to have having a few extra drinks around Christmas.

This is one of two common misunderstandings about epidemiologic information, namely that it contains some moral judgment. If you drink X units of alcohol, you can look on a ginormous chart and see your group’s average outcomes on a range of variables, including heart attacks, longevity, accidents and so forth. The data don’t care if you live more or fewer years, yet many people look at them as a moral compass or rebuke.

The other widespread misunderstanding of epidemiologic information is the conflation of population averages with the outcomes of individual cases. When the U.S. Preventative Services Task Force announced that routine mammograms could be delayed until a woman is age 50, I knew they were going to get eviscerated in the media by powerful individual stories, e.g. “I had a mammogram at age 41 and that’s why I am alive so how dare you put out guidelines that would murder women like me!”. Trying to explain a complex epidemiologic conclusion in the press while being challenged by a vivid, easy to understand individual story is absolutely hopeless.

Because public policy decisions often have to be guided by epidemiologic information, some people have called for better science education in K-12 so that more Americans can understand epidemiology. A noble goal, but probably a quixotic one. In popular discourse, we tend to get sucked in by individual stories, and ’twill always be thus. For me this is just a case of why it is good to have a republic and not a democracy.

Author: Keith Humphreys

Keith Humphreys is the Esther Ting Memorial Professor of Psychiatry at Stanford University and an Honorary Professor of Psychiatry at Kings College London. His research, teaching and writing have focused on addictive disorders, self-help organizations (e.g., breast cancer support groups, Alcoholics Anonymous), evaluation research methods, and public policy related to health care, mental illness, veterans, drugs, crime and correctional systems. Professor Humphreys' over 300 scholarly articles, monographs and books have been cited over thirteen thousand times by scientific colleagues. He is a regular contributor to Washington Post and has also written for the New York Times, Wall Street Journal, Washington Monthly, San Francisco Chronicle, The Guardian (UK), The Telegraph (UK), Times Higher Education (UK), Crossbow (UK) and other media outlets.

12 thoughts on “The Hopelessness of Epidemiologic Understanding”

  1. I don't disagree with your basic premise, but I would be interested in seeing the supporting data for the particular UK guidelines.

    I'm by no means an expert on this, but NIH's Health Benefits and Risks of Alcohol Consumption states that the

    mortality curve in countries with reasonably high rates of CHD is J shaped and that:

    Men who averaged 30 grams of alcohol (two drinks) per day had the same mortality as abstainers, whereas a significant increase in mortality was found for those consuming at least 40 grams of alcohol per day.

    The UK guidelines are measured in "units" of 8g, and they state:

    Men should drink no more than 21 units of alcohol per week (and no more than four units in any one day).

    In other words, the "no more than" limit translates to significantly less (24g/day) than the limit at which your risk is

    more or less the same as that for abstainers and the daily limit is only slightly more than the risk-neutral daily

    consumption rate, which leaves me a bit confused. I'm especially interested in the basis for the low amount

    for single-day consumption. I've seen this kind of recommendation a number of times but I've never been

    able to find a source that supported it.

  2. EKR: UK and US drinking guidelines do differ, and this was a needle our Translantic authorial team tried to thread when we wrote The Treatment of Drinking Problems (e.g., should a pregnant woman drink nothing per the U.S. advice, or is a little alcohol okay per the U.K. advice?). Part of this is cultural differences in views of alcohol but much of it has to do with the different comparison groups on has in the two countries. Abstainers in the U.S. are a large group about which we can make reliable estimates. Abstaining is much more unusual in the U.K. and there is also more heterogeneity within that group (e.g., It is a good assumption that a US abstainer is a regular religious service-goer, which has many health implications, that is not a good assumption in the U.K., people abstain for a wide range of reasons and those reasons often correlate with health-related factors).

  3. Keith,

    Pardon me if I'm being unusually dense here, but I'm not sure I fully understand what you're saying here.

    As I understand the situation, there is some available data on the impact of alcohol on health which is to some

    extent confounded by different alcohol consumption patterns in various cultures. Similarly, when that data is

    used to produce recommendations, that needs to be filtered through cultural factors. Do I have that correct

    so far?

    Let's take the concrete case of alcohol consumption during pregnancy. Is your argument that the optimal

    amount of actual consumption (ignoring the effect of complaince) is different between cultures or merely

    that cultural factors impact how much compliance you're likely to get out of a given recommendation?

    To turn to the second case, is your argument that the optimal rate of consumption in the UK actually

    differs from that in the US or merely that cultural factors make it hard to sort out the confounding factors

    in order to make that assessment?

  4. Statistical blindness isn't peculiar to epidemiology: it's part of the human condition, as Daniel Kahneman showed.

    Consider this experiment cited in Kahneman's Nobel prize lecture (follow the link). Subjects were shown this description of a woman called Linda:

    "Linda is 31 years old, single, outspoken and very bright. She majored in philosophy. As a student she was deeply concerned with issues of discrimination and social justice and also participated in antinuclear demonstrations."

    and a list of eight possible outcomes describing her present employment and activities. The two critical items in the list were #6 (“Linda is a bank teller”) and the conjunction item #8 (“Linda is a bank teller and active in the feminist movement”).

    Pretty much everybody, including (dixit Pinker) graduate students with formal statistics training, ranks 8 as more probable than 6, because "feminist" screams a pattern match; but it's impossible. The joint probability must always be lower.

    Emotional engagement, as with health risks and climate change, just builds on these inherent hunter-gatherer cognitive biases. (My hunter-gatherers, not Kahneman's, but I like them here).

  5. EKR

    Sorry if I was too cryptic on round one. Odds ratios in epidemiology always depend on a contrast group. In alcohol studies it is typically abstainers. Because we have a large number of abstainers in the U.S. with homegenity on a number of health habits we get a different answer here when we ask "How does risk change if someone drinks X drinks a week " than we would if we asked the same question in the UK where few people abstain and do so for more heterogenious reasons health wise. For example, a higher proportion of abstainers in the UK abstain because they have poor health, whereas here we have a higher proportion of healthy young people among the abstaining population who abstain for moral/religious reasons. It is thus possible that the same level of drinking will be relatively safer in the UK (when compared to the more sickly population of abstainers) than in the US (when compared to a more healthy group of abstainers).

    Culture definitely then influences how bad people think the risks are — i.e. Whether giving up alcohol during pregnancy is worth a .25% decrease in the risk of a birth defect. But that isn't about data or science any more, it's about judgments of utility which are subjective.

  6. Keith,

    Statistics is a wholesale science, people want retail. I stress this to my Statistics students, and some of them get it. One way to tell those who get it is that when they figure it out they don't much like it. I hold a doctorate in Statistics, and when it comes to my life I don't much like it, either. The knowledge I want is what my outcome will be: the best Statistics can do is tell me what risk group I'm in.

    Like you, when the announcement on mammograms came out I knew they were going to be skewered. The timing was awful (IIRC Sarah Plain and Uninformed had just been on another Death Panel rant) and people don't understand Statistics well enough to follow the logic. My wife (a very smart woman who teaches technical writing) was one of the livid people. Even after I explained it to her, she persisted in conflating individual outcomes with population rates.

    What finally worked was backing through Bayes' theorem with a neutral example: AIDS. There's no point in testing people without some risk factor for HIV infection because the rate in the general population is so low that a positive ELISA test acts only to raise the odds on infection. It does anything but make it a foregone conclusion.

    And so with mammography for women without identifiable risk factors. A women who is among the worried-well should probably get a mammogram for her own peace of mind. I don't know how we are going to "bend the cost curve down," without an ongoing program of reviewing screening recommendations.

  7. Dennis,

    >There’s no point in testing people without some risk factor for HIV infection because the rate in the general population is so low that a positive ELISA test acts only to raise the odds on infection. It does anything but make it a foregone conclusion.

    I don't quite follow the statistical point you are making here. Could you elaborate a bit for a fairly numerate non-professional?

  8. "There’s no point in testing people without some risk factor for HIV infection because the rate in the general population is so low that a positive ELISA test acts only to raise the odds on infection."

    But a negative ELISA means you don't have HIV, right? Which could be interesting to, for example, an insurance company. I had to take an HIV test before I got life insurance.

    I don't understand the point about whether pregnant women should drink, though. Whatever the danger of drinking while pregnant, it's the same whether the pregnant woman is in the US or in England. If a pregnant American woman moves to London, the risk to her baby if she drinks, whatever that risk is, stays the same. So how can the advice change?

  9. Cardinal Fang: Will try this one more time. These are odds ratios, they are all about relative risk.

    Let's take an extreme case to make the point. In country #1 all pregnant women drink a little during pregnancy, with the exception of those whose AFP test is abnormal in the first trimester. These women are ordered by their doctors to never drink and they all obey. When you look at the outcomes of moderate drinking pregnant women in that country versus that of abstainers, you see, wow, the abstainers had more birth defects and dificulties in delivery. In country #2, no woman drinks during pregnancy, except for those with an abnormal early test who drink moderately under the incorrect belief that moderate drinking is good for a baby with a discouraging AFP test, and the results reverse because the comparison group has changed.

    In the eyes of God of course, alcohol does what it does to a foetus wherever you live, but my post was statistical and not theological in nature.

  10. Keith,

    Based on your two points, it seems to me that the issue you're raising is that there is confounding

    from other behaviors. But, as you say, the effect of alcohol on the fetus is constant, so the problem

    at hand isn't that we should be giving different advice in different countries but rather that it's

    unclear what the advice ought to be because the confounding factors have made it difficult to

    interpret the epidemiological evidence. But if we were able to interpret it correctly (i.e., if we

    were able to control for the other factors) then we would be able to give the correct advice and

    it would be uniform across the relevant populations. IAm I wrong?

    So, turning back to the question of recommended drinking levels for the non-pregnant, given your

    comments it seems like the UK data is likely to underestimate the advantage of abstaining compared

    to the US data (because the UK abstainers are comparatively sicker). (It's possible that the US

    data overestimates it if the abstaining lifestyle correlates with other healthy traits). Does the UK

    data show a higher benefit of abstaining than the US data regardless of this systematic bias?

    If so, are you arguing that it represents a real effect and not just confounding? If not, wouldn't

    the best plan be to take the most unbiased data you can get, which your arguments suggest

    is likely to be the US sample, potentially after trying to control for obvious other effects.

  11. Yes EKR you said the point better than I did, re: confounding. I agree with you of course that an unbiased estimate would be best but with the alcohol and pregnancy issue we will never have that because we can't do a controlled trial.

Comments are closed.