The Non-Mystery of Most College Ranking Systems

Anxious parents and their high school-aged children are a big market for university ranking systems. Appearing in a range of books and magazines, ranking systems promise to give the inside scoop on which universities are the best as determined by experts and complex formulae. Armed with this knowledge, you will supposedly not endure the shame of having a child in a university that is ranked only 24th, while your better informed next door neighbor launches his or her offspring into an institution that is ranked 23rd (or maybe even higher!).

But before you plunk down your hard-earned dosh for one of these guides, try something simpler and cheaper: Look at this list of universities with financial endowments over $1 billion. Despite all the allegedly complex equations and inside info, most college ranking systems don’t tell you much more than does this list.

For example, the first 34 national universities on the list of US News and World Report’s well-known system are all in the billion dollar club. The precise amount of endowment among the billionaire institutions doesn’t correlate perfectly with the specific ranks assigned by USNWR. A few of the billionaire universities are ranked a bit lower than their endowments would suggest (e.g., University of Michigan at Ann Arbor, New York University), and a few are a bit higher (e.g. California Institute of Technology). But broadly speaking, if a university is in the billionaire’s club, it’s going to be well-rated in most supposedly sophisticated ranking systems and if it isn’t, it will not be. The only exceptions to this rule are systems that rate universities in non-traditional ways, for example on gender equity, racial diversity or service to the public.

One could argue whether this is because rich universities are in fact better or because reputation often follows wealth in America (which it didn’t always, age of university used to matter much more). It’s probably a bit of both. But in any event purchasing a book or magazine to learn its expert ranking system is usually a waste of money.

Author: Keith Humphreys

Keith Humphreys is the Esther Ting Memorial Professor of Psychiatry at Stanford University and an Honorary Professor of Psychiatry at Kings College Lonon. His research, teaching and writing have focused on addictive disorders, self-help organizations (e.g., breast cancer support groups, Alcoholics Anonymous), evaluation research methods, and public policy related to health care, mental illness, veterans, drugs, crime and correctional systems. Professor Humphreys' over 300 scholarly articles, monographs and books have been cited over ten thousand times by scientific colleagues. He is a regular contributor to Washington Post and has also written for the New York Times, Wall Street Journal, Washington Monthly, San Francisco Chronicle, The Guardian (UK), The Telegraph (UK), Times Higher Education (UK), Crossbow (UK) and other media outlets.

9 thoughts on “The Non-Mystery of Most College Ranking Systems”

  1. With blogs, some astute posts don’t get much commentary because they are correct and complete. I think this might be one of those.

  2. NY f*ck*ng U is #33? Omigod!

    (For those of you not in the know, NYU is a mediocre university with an excellent law and medical school, and I’ve heard a pretty good film school. It–along with Boston University–pioneered the trick of jacking up rack rate tuitions to look more selective, and discounted like crazy if a promising undergrad just happened to show up.)

    I agree with politicalfootball’s assessment of Keith’s blog post, although perhaps not his conclusion.

    1. Well, I am in the know, and your information is a couple of decades out of date. NYU used to be rather mediocre, but has improved a lot over the last 20 years. Math is truly outstanding, so is the business school, and a bunch of other programs.

      As for ranking versus endowment, I suspect a better fit would look at both total endowment and endowment per student. Michigan and NYU obviously have more students than most other schools on the list, and thus have less endowment per student than others.

      Or, we could just assign a little less importance to rankings. There are many schools that (can) give you an excellent education, so stop obsessing about getting into the top 10.

  3. Agree with politicalfootball, too (and Ebenezer). It is also worth pointing out the the US News rankings were (and are) nothing more than a marketing gimmick that allowed the distant third of three national news magazines to survive. And very well, beyond it wildest imaginings it would seem. Unfortunately they also initiated a never-ending and largely futile progression of institutions trying to figure out how to move up, so to speak.

  4. Wouldn’t the endowment per student be a better metric? (It seems to capture more of the small elite institutions, and the leave out the state landgrant systems like Texas A&M.)

  5. I guess the next question would be, what then, actually goes into making a great school? I went to city college and state schools, so I admit I have no clue. 🙂

  6. I suspect the correlation is actually causation — rich schools can afford to game the ratings systems.

  7. 2 comments:
    1) IIRC, a paper was published in the journal, “Academic Medicine” about ten years ago, exploring the methodology of the USNWR’s rankings of medical schools. Again, IIRC, a large part of the data they based their rankings on was based on questionnaires sent out to the various medical school heads. The return rate of these questionnaires was around 25% or less, which has a marked negative effect on the reliability of the results–aside from the obvious tendency of Deans or Presidents to rank their own school highly, there is a lack of statistical validity when the return rate is this low.
    2)I have yet to find a physician or academician who could give a good account of what ‘excellence’ is, in a physician, or a medical school. If you can’t define what you are trying to measure, how can you rate institutions on its production?
    Worthless data does not magically become statistically meaningful if you analyze it to four significant figures.

Comments are closed.