January 9th, 2013

Aaron Ellis gives every single proximate reason why Notre Dame’s football team was completely crushed by Alabama in the BCS Championship Game. But he misses the higher-order cause: They were over-rated in the polls and did not deserve to be in the game in the first place.

After the election, there were welcome efforts by worthies such as Brad Plumer and Andrew Sullivan to hold political prognosticators accountable for ridiculously errant electoral predictions. The same thing should be done with the people who rank college football teams.

This isn’t just about the incredible lameness of this year’s championship game, which a Martian would have mistaken for a match between an NFL team and a high school junior varsity squad. Getting into the championship is a multi-million dollar reward for universities. Teams like Notre Dame, who have a national fan base that will buy tickets and merchandise and are a “name” are frequently over-rated and thereby get into big money bowls which they don’t deserve. If a team such as Boise State or the University of Wyoming had played the same teams as Notre Dame with the same results, they would not have been ranked as highly and would have been shut out of the big money.

Given that college football poll votes are documented and conducted weekly, it would not be hard to implement a system that rewarded performance at ranking teams. To take a completely bad apples approach, every time a team ranked in top 5 lost to an unranked team by 28 points, you could remove the voting rights of all the people who ranked the former team so highly. Or you could let the people who voted unsuccessfully for the unranked team have their votes count double in the following weeks. Or both. One could tinker with a range of solutions; any that rewarded performance and punished ineptitude (and self-promotion and venality to the extent that drives some individuals’ inaccurate rankings) merits consideration.

With no accountability, some teams will have an unfair, lucrative advantage and people who make lousy rankings and thereby generate lousy bowl games will keep getting away with it.

p.s. To anticipate a weak objection: “But Alabama would have beaten anyone, they were so good.” Remember, they lost to Texas A&M in November and beat Georgia by only 4 points for their conference title in December. And Notre Dame didn’t just lose, they were utterly humiliated in all phases of the game. If Coach Nick Saban had been in a mean mood, the final score would have been 70-7. There were much better options available, and the poll voters who generated this one should be held accountable for their failure.

Tags: , , , ,

25 Responses to “Will We Hold (Football) Pollsters Accountable?”

  1. Mark Kleiman says:

    I am shocked – shocked! – that some of the universities making money by inflicting brain injuries on their students are getting more of their share of the loot while others get less, and horrified that as a result admirers of gladiatorial combat are sometimes displeased with the sport offered. Too bad the fans don’t have the Imperial option of having the losers killed.

  2. John M says:

    I find myself looking over my shoulder and suspecting that this is some sort of Jonathan Swift-style parody that is flying over my head. If so, congrats. I’ll take the hook out of my mouth. If not, this is a really silly and sloppy post, quite comically bad. Notre Dame was the only major program that was undefeated and eligible for the postseason (Ohio State was 12-0 but on probation). Further, you seem to be hold the mistaken belief that the human polls alone determine who plays for the national championship. In fact, the BCS standings include both human polls and computer ratings that are designed to add an element of objectivity. Notre Dame was ranked #1 by both of the human polls and by every single one of the computer ratings that is part of the BCS. Again, Notre Dame was not merely #1 in an average of the computer ratings; Notre Dame was #1 in every single computer rating.

    You say that there were better options than Notre Dame. Name them? Oregon? Maybe. But Oregon lost to Stanford, and Stanford lost to Notre Dame. Kansas State? Maybe. But Kansas State lost to Baylor, which lost to Oregon (a team that both Notre Dame and K-State beat on the road).

    Nearly everything that you say in this post is wrong, other than your description of how thorough Alabama’s victory was. No arguing with that. But I think there are two main points that are your most fundamental errors:

    1. Football games are not as predictable as you imagine. Sometimes a team that legitimately is among the top 5 in the nation falls apart in game. For instance, on November 17, undefeated and top-ranked Kansas State played an unranked Baylor team that was 4-5 hadn’t defeated a decent team all year. It wasn’t even close. Baylor won by 28 points. In that case, every single pollster would have lost his or her vote. And why? Nobody got it wrong. There was simply no discernible reason before that game to believe that Baylor would compete with Kansas State, let alone run away with it. This isn’t a bug. It’s a feature. Shocking upsets like that are why people like sports!

    2. Somewhat related, I don’t think you understand the purpose of the BCS. The purpose is not to place the two teams that would be the two best team if the season were played 100 times. It’s designed to match up the teams that had the best seasons as it happened. Notre Dame had something of a charmed existence in 2012, but ultimately finished undefeated against a quality schedule, and did well enough that all of the major computer rating systems had the Irish #1. Again, if you have developed a better objective way of measuring the quality of college football teams, then by all means describe it. Otherwise, your outrage is pretty hollow.

    • Seitz says:

      Not only did Notre Dame go undefeated, but they played a schedule that included five teams ranked in the pre-season top 25. Now, as it turned out, USC, Michigan, Michigan State, and Oklahoma were all overrated, and ND got Stanford before Stanford realized how good Stanford could be. But their schedule included more teams that pollsters, going into the season, expected to be very good. They were remarkably lucky, but they scheduled tough, and they beat every team put in front of them.

      One can argue that they were not one of the top two teams in the country, and certainly one would have received a lot of support for that argument going into the NC game. But in every American team sport, the two teams that play for the championship get there by earning it through their play, regardless of whether they’re one of the consensus top two teams. Notre Dame earned it on the field.

    • toby says:

      I tend to agree with you … excellent example in Rugby Union last month. New Zealand consistently produces the best ranked team in the world, but has a poor enough record in World Cups. However, they won the World Cup last year, and had a record winning streak up until last month. England, by conrast, had a almost totally new team ranked about 6th in the World. No one gave them a chance against the NZ invincibles, but they won by a record margin.

      There are imponderables in sport that do not affect politics – mental preparation, exhaustion at the end of a long season, pure dumb luck, a pass that goes to hand or goes astray (sounding like Al Pacino, now :) ). Even the greatest teams have suffered unexpected collapses. Barcelona FC with Lionel Messi are (arguably) the best soccer team in the world, with the world’s greatest player. They win La Liga, but have only managed to win the European Champions League once in the past few years, losing to lesser teams like Inter Milan and Chelsea.

  3. Ken D. says:

    FWIW, this site — http://www.masseyratings.com/cf/compare.htm — aggregates a staggering number of college football polls, nearly all of them computer/formula based. Before the championship game, the average of 132 polls had Notre Dame number one by a sliver over Alabama. The bottom line is that there is rarely one best team, and there is no perfect or ideal to calculate the one, or two, or four, best teams. Let the argumentation continue.

  4. Conspiracy theory says:

    Obviously, Keith doesn’t know what he is talking about.

    Still, an 8 team playoff would solve whatever problems any sort of problems a ranking system might have.

    But it still wouldn’t solve the injury problem.

    • Seitz says:

      Actually, it could help the injury situation if the effect was to back to an 11 game season. You add a couple extra games for a few team,but reduce the overall number of games played. Big if, though.

    • Quercus says:

      >Still, an 8 team playoff would solve whatever problems any sort of problems a ranking system might have.
      Because there would be absolute universal agreement on who would be #8 and who would be #9, right?
      And, as SuperBowl history has proved, a three round playoff will of course eliminate any possibility of a lopsided championship game.

      Anyway, John M gets to the inherent contradictions of ‘ranking’ teams without specifying whether that’s supposed to rank by accomplishments (in which case before the final game ND was pretty clearly #1: they went undefeated against a schedule as strong as anyone else), or predicting future performance (in which case, even before the game, the general consensus was that Alabama was significantly more likely to win). I think this is the root of a lot of controversy over college football polls; though of course, since it’s all about entertainment, nobody really wants to clarify things enough to make the controversy go away.

      Finally, I don’t agree that Bookies are trying to calculate results accurately. They don’t make money by winning bets; they make money by using winning bets to pay off losing bets (ideally perfectly balanced) and taking a percentage off the top. So ideally they want to set their odds so that an equal amount of money is bet each way, not necessarily what they think are actual odds, and they’ll change their odds if too much money is bet one way or the other. So really, what bookmakers lines give you is some rough total of what the betting public thinks is the right line, but partly washed out by all the non-rational bets people make.

  5. Paul says:

    There really is only one way to properly rate teams in contests such as the one we’re discussing here. And the thinking is similar to that used for the Bermuda Triangle myth. Regardless of how many anecdotal events are attributed to the Bermuda Triangle the only one that can be said to have some empirical value is the old tried and true, “follow the money.” If there were really some statistical evidence showing The Triangle had a disproportionate number of accidents then the insurance companies would charge a premium commensurate with the demonstrable evidence……..They don’t.

    Likewise, with football games, or any other sporting event. The only source with any real credibility that can demonstrate consistent accuracy in predicting the outcome of sporting events are the Las Vegas Sports Books. Those people have billions of dollars riding on the outcome of games. Obviously, they have figured out a way to calculate results with enough accuracy to remain profitable year after year. This year they calculated that Alabama was an eight point favorite over Notre Dame. In other words they risked millions of dollars of their own money on their prediction that Alabama would beat Notre Dame by more than a touchdown.

    All other polls were just prejudicial guesses. If you want to know the results of a game, “follow the money.”

    • Yes. What is the point of a poll on a sporting contest? I’d never actually heard of one before this. An election poll asks people what they intend to do about something, viz. vote one way or the other, and has some predictive value. A poll of RBC readers to name the hottest blogger would be worthless.

      • Josh says:

        But entertaining.

      • J. Michael Neal says:

        A poll of RBC readers to name the hottest blogger would be worthless.

        You say that only because we’d all pick Kleiman.

        • John G says:

          well, I might pick Wimberley, followed by Humphrey, but maybe that’s my Canadian tendency to balance mid-Atlantic (and I think John M’ post above has damaged Keith’s standing, at least in sports topics.)

          • Katja says:

            Let me be the first to point out that there are those of us who don’t consider football illiteracy to be a character flaw. :)

            (I’m saying this as someone who has relatives in Menominee, MI. Not quite Green & Gold Central, but still pretty close.)

          • Keith Humphreys says:

            And there are others of us who appreciate classy and thoughtful commenters — thanks Katja.

  6. Mitch Guthman says:

    Another solution, of course, would be to require academic institutions to field teams composed of genuine student athletes coached by people who are payed similarly to professors at that institution. As opposed to semiprofessional football teams, staffed by young men serving forced NFL apprenticeships and coached by professional coaches who are paid millions of dollars. In other words, why not bring back amateur football instead of what are basically the NFL’s farm clubs?

  7. doretta says:

    There is no system you can set up that will guarantee that you won’t get a rout in the final game. As has been pointed out, the computers picked Notre Dame too and they don’t know or care how large or fanatical their fan base is or how much money they have or how often they get to be on national TV.

    A playoff system won’t guarantee that you won’t have a rout either but the more teams the playoff includes the more satisfied people can be that the teams they think are the two best got a chance to prove it on the field.

  8. David says:

    I must say that I agree with Keith. Yes, the computers all ranked ND #1, but to me that indicates possible flaws in their ranking processes, not that ND deserved to be in the NC. For example, my understanding is that the computers are not allowed to take the margin of victory into consideration — is that a good rule?

    Also, the idea that ND deserved a place in the NC because they “earned it on the field” is ridiculous and the fact that they had a schedule that appeared strong before the season started is irrelevant.

    • Cranky Observer says:

      The fundamental text in this area is Arpad Elo’s “The Rating of Chess Players Past and Present”. Here’s a summary and discussion:

      http://www.chessbase.com/newsdetail.asp?newsid=4326

      I’d recommend that you try implementing the algorithm and operating it for a few seasons (perhaps using a large high school league as input). For use in sports with relatively low numbers of games (e.g. football) you’ll have to figure out how to set the initial rank ordering and the constants. It is not an easy problem; the people who run the major rankings models undoubtedly put a lot of thought into it and of course their results are often criticized in hindsight.

      Cranky

    • J. Michael Neal says:

      They don’t allow margin of victory to be considered because they don’t want to provide incentive to run up the score on weaker teams. A decade ago, in the early days of women’s ice hockey as a significant international sport, aggregate goal margin was used as a tiebreaker in tournaments. So you’d see the US or Canada beat a team like Kazakhstan 70-2. Yes, in a 60 minute ice hockey game. It was a disaster all around. I doubt that adding margin of victory to the computer rankings could add enough value to compensate for the downside.

  9. Lowell says:

    There are too many teams, and too few games, to be sure a deserving team isn’t missed out for the top slot.

    The BCS is a well-meaning attempt to make the season-snapshot view as meaningful as possible, but you have to keep the expectations reasonable. There are more than a hundred Division I football teams. We know that some (Ivy League, anyone?) have (little-to-)no chance at the top spot, but the number of teams to consider is easily two or three times larger than, say, the NFL. They can’t (realistically) play (significantly) more games than they do now, and some of those games are against teams that have no hope of reaching the top slot.

    There are not, and cannot be, enough data points to say which is the best college football team. Let’s be reasonable and be happy with (or unhappy without) the championship for a conference (league/whatever-as-long-as-less-than-two-dozen-teams).

    • Cranky Observer says:

      Agreed. I personally don’t understand the hunger for some sort of abbreviated football playoff; even basketball with its 68 team playoff generates a lot of unexpected “1-and-done” upsets which knock out highly ranked teams. I thought the historic bowl system as expanded was fine and the month of arguing afterward about who “should have been” the national champion was part of the fun.

      Cranky

      OK, the hunger comes from cable TV’s insatiable appetite for more football games and more cash, but I don’t find that a particularly attractive aspect of college sport (per Mitch Guthman at 10:02)

      • J. Michael Neal says:

        Agreed. I still can listen to Penn State fans say that they were screwed out of the 1994 National Championship by the poll voters. I try to point out to them that they have derived far more entertainment value out of not being named national champions than Nebraska fans have out of winning.

        They don’t buy that argument.


SiteMeter