Analysis, character, and chance

A wise man reflects on the value, and the limits, of analytical models.

Dr. Manhattan of Blissful Knowledge finds a gem from Bill James, the man who rewrote the book on baseball by applying serious statistical analysis to the question of what actually wins games. (I’ve snipped out some extraneous matter.)


I recently saw you quoted to the effect that veteran leadership had enabled the Red Sox to come back from down 0-3 in the ALCS. The immediate response was to doubt your sincerity: “Bill couldn’t mean that!”


I believe in a universe that is too complex for any of us to really understand. Each of us has an organized way of thinking about the world—a paradigm, if you will—and we need those, of course; you can’t get through the day unless you have some organized way of thinking about the world. But the problem is that the real world is vastly more complicated than the image of it that we carry around in our heads. Many things are real and important that are not explained by our theories—no matter who we are, no matter how intelligent we are.

As in politics we have left and right—neither of which explains the world or explains how to live successfully in the world—in baseball we have the analytical camp and the traditional camp, or the sabermetricians against the scouts, however you want to characterize it. I created a good part of the analytical paradigm that the statistical analysts advocate, and certainly I believe in that paradigm and I advocate it within the Red Sox front office. But at the same time, the real world is too complicated to be explained by that paradigm.

It is one thing to build an analytical paradigm that leaves out leadership, hustle, focus, intensity, courage and self-confidence; it is a very, very different thing to say that leadership, hustle, courage and self-confidence do not exist or do not play a role on real-world baseball teams.

Right! Right! Right! Right! Right! The map is not the terrain, and the model is not the phenomenon, in baseball or in politics or in policy analysis.

But note that both the interviewer and James assume that there was some reason the Red Sox came back from three games down to win the pennant: that if it couldn’t be explained analytical terms, then the cause must be some intangible.

The alternative view is that each game was an independent event, with the probability of either team winning somewhere near 50%, and that the Red Sox made up for eighty years of bad luck by having the coin come up “heads” four times running. That’s going to happen one time in sixteen, just by chance.

And yet the impulse to find some fundamental cause, whether that cause is “leadership” or something else, is nearly irresistible.

Author: Mark Kleiman

Professor of Public Policy at the NYU Marron Institute for Urban Management and editor of the Journal of Drug Policy Analysis. Teaches about the methods of policy analysis about drug abuse control and crime control policy, working out the implications of two principles: that swift and certain sanctions don't have to be severe to be effective, and that well-designed threats usually don't have to be carried out. Books: Drugs and Drug Policy: What Everyone Needs to Know (with Jonathan Caulkins and Angela Hawken) When Brute Force Fails: How to Have Less Crime and Less Punishment (Princeton, 2009; named one of the "books of the year" by The Economist Against Excess: Drug Policy for Results (Basic, 1993) Marijuana: Costs of Abuse, Costs of Control (Greenwood, 1989) UCLA Homepage Curriculum Vitae Contact: