The Non-Mystery of Most College Ranking Systems

Anxious parents and their high school-aged children are a big market for university ranking systems. Appearing in a range of books and magazines, ranking systems promise to give the inside scoop on which universities are the best as determined by experts and complex formulae. Armed with this knowledge, you will supposedly not endure the shame of having a child in a university that is ranked only 24th, while your better informed next door neighbor launches his or her offspring into an institution that is ranked 23rd (or maybe even higher!).

But before you plunk down your hard-earned dosh for one of these guides, try something simpler and cheaper: Look at this list of universities with financial endowments over $1 billion. Despite all the allegedly complex equations and inside info, most college ranking systems don’t tell you much more than does this list.

For example, the first 34 national universities on the list of US News and World Report‘s well-known system are all in the billion dollar club. The precise amount of endowment among the billionaire institutions doesn’t correlate perfectly with the specific ranks assigned by USNWR. A few of the billionaire universities are ranked a bit lower than their endowments would suggest (e.g., University of Michigan at Ann Arbor, New York University), and a few are a bit higher (e.g. California Institute of Technology). But broadly speaking, if a university is in the billionaire’s club, it’s going to be well-rated in most supposedly sophisticated ranking systems and if it isn’t, it will not be. The only exceptions to this rule are systems that rate universities in non-traditional ways, for example on gender equity, racial diversity or service to the public.

One could argue whether this is because rich universities are in fact better or because reputation often follows wealth in America (which it didn’t always, age of university used to matter much more). It’s probably a bit of both. But in any event purchasing a book or magazine to learn its expert ranking system is usually a waste of money.

Comments

  1. politicalfootball says

    With blogs, some astute posts don’t get much commentary because they are correct and complete. I think this might be one of those.

  2. Ebenezer Scrooge says

    NY f*ck*ng U is #33? Omigod!

    (For those of you not in the know, NYU is a mediocre university with an excellent law and medical school, and I’ve heard a pretty good film school. It–along with Boston University–pioneered the trick of jacking up rack rate tuitions to look more selective, and discounted like crazy if a promising undergrad just happened to show up.)

    I agree with politicalfootball’s assessment of Keith’s blog post, although perhaps not his conclusion.

    • tsts says

      Well, I am in the know, and your information is a couple of decades out of date. NYU used to be rather mediocre, but has improved a lot over the last 20 years. Math is truly outstanding, so is the business school, and a bunch of other programs.

      As for ranking versus endowment, I suspect a better fit would look at both total endowment and endowment per student. Michigan and NYU obviously have more students than most other schools on the list, and thus have less endowment per student than others.

      Or, we could just assign a little less importance to rankings. There are many schools that (can) give you an excellent education, so stop obsessing about getting into the top 10.

  3. KLG says

    Agree with politicalfootball, too (and Ebenezer). It is also worth pointing out the the US News rankings were (and are) nothing more than a marketing gimmick that allowed the distant third of three national news magazines to survive. And very well, beyond it wildest imaginings it would seem. Unfortunately they also initiated a never-ending and largely futile progression of institutions trying to figure out how to move up, so to speak.

  4. SamChevre says

    Wouldn’t the endowment per student be a better metric? (It seems to capture more of the small elite institutions, and the leave out the state landgrant systems like Texas A&M.)

  5. says

    I guess the next question would be, what then, actually goes into making a great school? I went to city college and state schools, so I admit I have no clue. :)

  6. oddpal says

    I suspect the correlation is actually causation — rich schools can afford to game the ratings systems.

  7. RE Garrett says

    2 comments:
    1) IIRC, a paper was published in the journal, “Academic Medicine” about ten years ago, exploring the methodology of the USNWR’s rankings of medical schools. Again, IIRC, a large part of the data they based their rankings on was based on questionnaires sent out to the various medical school heads. The return rate of these questionnaires was around 25% or less, which has a marked negative effect on the reliability of the results–aside from the obvious tendency of Deans or Presidents to rank their own school highly, there is a lack of statistical validity when the return rate is this low.
    2)I have yet to find a physician or academician who could give a good account of what ‘excellence’ is, in a physician, or a medical school. If you can’t define what you are trying to measure, how can you rate institutions on its production?
    Worthless data does not magically become statistically meaningful if you analyze it to four significant figures.