The Trials of British Psephology

The 2015 UK Election wasn’t just a nightmare for the Liberal Democrats and Labour, it was also a disaster for British pspehologists. In the closing weeks of the election, all the major pollsters repeatedly predicted a hung parliament, with the Tories and Labour netting approximately the same number of seats. In the wake of Conservatives’ stunning victory, pollsters have been defending their methods and trying to determine how they got the election outcome so wrong.

My initial reaction to the pollsters was sympathetic. 2015 was an usual year: There were more parties than ever with a credible shot of winning seats, the rise of Scottish nationalism was a novel and powerful force, and the number of homes still using ye olde telephone landline was at an unprecedented low. But then I found the video below, showing that in a much simpler election landscape the pollsters muffed the catch in eerily similar fashion in 1992, again failing to predict the Conservative majority that the voters delivered.

I don’t claim to know the full explanation for these polling errors, but this candid admission by Damien Lyons Lowe, head of Survation, suggests that rational herding is part of the problem. This is his description of the poll Survation completed on the eve of the election:

Survation Telephone, Ballot Paper Prompt:
CON 37%
LAB 31%
LD 10
Others (including the SNP) 6%

Which would have been very close to the final result.

We had flagged that we were conducting this poll to the Daily Mirror as something we might share as an interesting check on our online vs our telephone methodology, but the results seemed so “out of line” with all the polling conducted by ourselves and our peers – what poll commentators would term an “outlier” – that I “chickened out” of publishing the figures – something I’m sure I’ll always regret.

Expert predictions of the outcome in King vs. Burwell

The week before King v. Burwell was announced, I confidentially polled an elite group of health policy and legal experts to get their personally-assessed prior probability that the King plaintiffs challenging ACA subsidies would actually win this case. The somewhat-arbitrary group included leading health economists, political scientists, journalists covering the case. It also included several legal antagonists who had written briefs on the various sides.

I received more than forty responses. Not surprisingly, given my own liberal views, about three-quarters of the responses were from Democrats. Although Democrats and Republicans had wildly different views of who should win, the distribution of predicted probabilities of a plaintiff victory was surprisingly similar between the two groups. Amazingly, Cato’s Michael Cannon and I offered identical prior predictions—20% probability of plaintiff victory.

Among Democrats, the median predicted probability of plaintiff victory was 40%. Among Republicans, the median was 37.5%. The means showed a bit more of a difference between the two groups: 46% among Republicans, 37.5% among Democrats. Yet five of the six highest predicted probabilities of a plaintiff victory came from Democratic experts; two were 70% or higher.
The overall distribution is shown below.

clunky expert poll results

Judging by my admittedly- clunky informal poll, the Obama administration’s emphatic victory surprised many close observers on both sides. Many Democrats reported that the plaintiffs had more than 0.5 probability of winning. Otherwise, why would the Supreme Court have jumped to take the case? We may never know the answer to that question.

We don’t merely know this from polls. Hospital Corporation of America stock rose 8% at the decision’s announcement. (See below from That is an amazing jump, which reflects hospitals’ strong stake in ACA’s effective national implementation.

HCA stock price (

Many states went to considerable trouble devising contingency plans, suggesting that they, too, took this case very seriously indeed.

After the ruling was announced, I tweeted that conservatives magnified their political and legal losses by tethering themselves to a preposterous legal case. I stand by that deep, 140-character argument. Yet it is definitely an ex post argument. The smart money believed that the plaintiffs might well have won.

We’re likely to see a huge amount of hindsight bias applied to this case. From the plaintiff’s perspective, this seems ex post like a more doomed tactical gambit than it actually was, when properly viewed from an ex ante perspective. From my opposing perspective, this is sobering. Fortunately, the same insights make the government’s emphatic victory appear more exhilarating and important than it would otherwise seem to be.

Philip Roth as Founding Father

The greatest trolling exercise in the history of health policy is now over.

My short take for Politico on the King case.

The Supreme Court has finally spoken. It never really needed to speak at all. The Court’s 6-3 decision to uphold federal subsidies under the Affordable Care Act was simple and emphatic, written by Chief Justice Roberts, no less. Justice Antonin Scalia’s dyspeptic dissent indicates the extent of the administration’s legal victory. Stock prices for the Hospital Corporation of America jumped approximately 8 percent with this decision, providing some sense of the economic havoc that might otherwise have ensued.

I’m gratified by the outcome. But I remain saddened by the full history of this case….

More here.

It’s a shame that the best health policy minds in both parties spent many months battling over this preposterous thing. We could have been trying to make better policy. IBy tethering themselves to King and then suffering a comprehensive legal defeat, conservatives ironically set back their own efforts while further embedding ACA within the fabric of American life.

There are many ways to comfort the grieving in this life

This isn’t a bad one.

Not a bad bit of speech-writing, either.

As a nation, out of this terrible tragedy, God has visited grace upon us, for he has allowed us to see where we’ve been blind. He has given us the chance, where we’ve been lost, to find our best selves. We may not have earned it, this grace, with our rancor and complacency, and short-sightedness and fear of each other — but we got it all the same. He gave it to us anyway. He’s once more given us grace. But it is up to us now to make the most of it, to receive it with gratitude, and to prove ourselves worthy of this gift.

Weekend Film Recommendation: Pather Panchali [Song of the Little Road] (1955)

Originally trained as a visual designer, Satyajit Ray is regarded in Indian cinema history as one of its greatest directors, and as the auteur responsible for creating the first domestic film to acquire success abroad. That film, which is the first of both Ray’s career and of a trilogy named after its protagonist “Apu,” is Pather Panchali [Song of the Little Road] (1955).

While the sequels deal with Apu’s life as an adolescent and then as a young adult, Pather Panchali centers on Apu’s birth and early childhood. Born to a penurious family in rural Bengal, the father Harihar (played by Kanu Banerjee) is a scholar, poet, and priest who is perennially trying to scrounge together some money to repair the family home, feed the children (Apu is accompanied by an older sister named Durga, played exquisitely by Uma Dasgupta), and purchase new clothes for his wife Sarbajaya.

Screen shot 2015-06-26 at 01.10.05

Although the trilogy is named after Apu and follows his life over multiple decades, it is his mother Sarbajaya, played by Karuna Banerjee, who is really the focal point for Pather Panchali. At the beginning of the film, she is the proud guardian of the homestead, delegating responsibilities and keeping the children fed. Even though she is understandably hurt when she overhears her neighbors gossiping unflatteringly about her parenting, Sarbajaya is a composed and dignified woman. As the story progresses, however, and the family’s poverty deepens, it’s through her quiet frustrations and increasingly exasperated efforts to pawn belongings that we notice the growing intensity of the family’s despair, and the gradual weakening of the formidable determination she showed when we first met her. Apu’s upbringing is thus told through his relationship with his mother, who—although always loving— is over-bearing, stern, and disciplinary.

Sarbajaya’s efforts to keep Apu on the straight and narrow are repeatedly flummoxed by Harihar’s cousin, the elderly Indir (played by Chunibala Devi). To Sarbajaya, Indir is a complete pest: she encourages the children to misbehave, she completely over-stays her welcome, and she has an unsettling fondness for morbid folk-tunes that enervate Sarbajaya’s spirits. All the same, Indir has an unusual charm. Her meditations on death and life border on insincere attention-seeking, but in her absence the other characters learn that Indir has the most to teach.

Apu’s outlet for childish mischief is his relationship with his older sister Durga, who shares with Apu her curiosity about the daily locomotive that passes near their home. Ray’s fawning representation of the train and the associated industrialization of India synchronized with Nehru’s platform of non-alignment, as Pather Panchali evokes a welcoming of new-ness and a readiness to participate in the new world of industrial advance and technological progress. The congruity between Ray’s thematic delivery and Nehru’s political agenda secured not only the funds necessary to complete filming, courtesy of the West Bengal government, but it also earned Nehru’s endorsement when Ray took the film abroad to Cannes.

Screen shot 2015-06-26 at 01.12.06

Rather than assigning a leitmotif to each character, Ravi Shankar’s soundtrack assigns a classical raga that corresponds to each emotion or impression conveyed over the course of the film. The music sometimes follows the mood of the film, and at other times it leads the audience’s impression. The pace obeys none of the forms that modern movie-going audiences have come to expect, and so some may find the film slow and even possibly tedious, especially during the first half. But Pather Panchali rewards patience, as the story gathers weight and Ray’s vision becomes clear.

Rather than post a trailer, here’s a clip of Apu and Durga playing in the rain that gives a flavor of the film’s mood.

The GOP vs. the Reality Principle

If you, like me, have a twisted sense of humor and limited compassion, it’s rather funny to see all the Republican politicians and conservative pundits decompensating in the face of the rather predictable outcome of King v. Burwell

Not that it’s unreasonable for them to be disappointed that the court didn’t once again make up legal principles out of whole cloth in order to serve right-wing prejudices and policy preferences. But the reaction isn’t the “Well, it was worth a try” that might be expected when a long-shot didn’t come in. It’s sheer uncomprehending grief and outrage. Instead of saying “Too bad, I guess John Roberts decided he couldn’t stretch reality far enough to let us win this one,” the right wing is saying “John Roberts betrayed us! He was never really a conservative!”

More than anything else, it reminds me of Karl Rove on Election Night 2012, insisting that Barack Obama hadn’t really have carried Ohio (which, when all the votes were in, he carried by a reasonably comfortable 3 percentage points) and thus the Presidency.

In 2012, then The Republicans had spent so much time “unskewing” the polls that they’d really convinced themselves that Romney was going to win. Recall that Romney was so confident he hadn’t even drafted a concession speech.  This time, they’d convinced themselves that the fundamentally frivolous challenge in King - asserting that the Congress had deliberately set up Obamacare to fail – was obviously valid. (As Roberts points out in a footnote, Scalia’s dissent in NFIB, the previous Obamacare case, specifically asserted that the subsidies formed an essential part of the law.)

So when John Roberts, in effect, called “bullsh*t,” Fox News viewers, including the pols and the professional chatterers, were utterly blindsided, and reacted by insisting that the problem was with Roberts rather than with the plaintiffs’ case.

In a purely partisan sense, it’s an advantage for Democrats to be running against a fundamentally unhinged party. But from a patriotic viewpoint, the realization that the country no longer has two political parties that can be trusted with the reins of government is pretty damned scary. It’s possible that a convincing Clinton win and a Democratic recapture of the Senate in 2016 will shock the GOP back to reality. But I wouldn’t bet on it.  Feeding right-wing fury is a profitable venture financially, and it works well enough electorally in off-years to keep the hustle going. My guess is that it will take a Clinton re-election landslide in 2020 to do the job.




King v Burwell office pool

Here’s my card. I’m not a lawyer, but do you really think this is about the True Meaning of the Law, and not politics?

King to lose 6-3. Majority: Breyer, Ginsburg, Kagan, Kennedy, Roberts, Sotomayor. The opinion, written by Roberts or Kennedy, will be on narrow grounds that the plaintiff’s reading would amount to unconstitutional federal coercion of the states, precedent NFIB v Sebelius. So it will duck the statutory construction issue.

Dissent by Alito, Scalia, and Thomas to uphold, because Obama. Scalia’s contortions on statutory construction, contradicting his own previous opinions, will be fun to read.

Dissent Concurring opinion [if I don't fix this I will never hear the end of it] by Sotomayor and Ginsburg listing about 200 reasons why the suit is frivolous, starting with the absence of standing. I’m not so confident about this; Roberts may have extracted their moderation as the price of his vote. But Sotomayor at least was very angry on the record about the Wheaton College double-cross after Hobby Lobby:

Those who are bound by our decisions usually believe they can take us at our word. Not so today.

Payback time maybe.

Your counter-offers?

How Much De-Incarceration is Enough?

Kevin Drum is surely correct when he argues that the U.S. puts way too many people in prison. With the imprisonment rate falling for five straight years and state after state passing major reforms designed to reduce the number of people behind bars, it’s a good time to ask precisely how much de-incarceration reformers need to declare victory.

One possible answer is to strive for a return to the rate of incarceration that prevailed historically until the mid-1970s. Until that point, the imprisonment rate rarely ventured too far in either direction from 0.1% – less than a quarter of the rate we have today. However, that standard was established before the modern feminist movement forced the criminal justice system to take violent crime against women seriously. There are currently a couple hundred thousand men behind bars for raping, assaulting and otherwise terrorizing women. We should not be nostalgic for the era when many or most of them would have gotten away with such crimes. This same point should be borne in mind by anyone who is tempted to look covetously at the low incarceration rates of countries such as India where violence against women almost never results in arrest and prosecution.

Would dropping incarceration to a Western European level be a reasonable goal for the U.S.? Certainly we should move in that direction, but the U.S. is more violent than any of those countries. Our homicide rate is about four times that of the U.K. and about ten times that of France and Germany. It is unrealistic to expect therefore that our incarceration rate will ever drop to their level.

An alternative way to approach the problem is to forget about comparative numerical targets and return to first principles: What is prison for? Typically, we send people to prison for one or all of three reasons (1) to keep the public safe, (2) to provide rehabilitation and (3) to punish people appropriately for things we think are wrong. The current level of incarceration is indefensible on those grounds, but we can strive to return to a level that is.

Kevin notes for example that while we are past the point where increasing incarceration keeps reducing the crime rate, such a point clearly exists, and we could find it again. A further important standard is when overcrowding has eased enough that prisons can again engage in meaningful rehabilitation versus, say, triple bunking inmates in what used to be the basketball court. Finally, when long sentences are again reserved for those who have committed truly serious crimes, we will know that the size of the prison population reflects a commitment to punishing criminals proportionately rather than indiscriminantly.