I’d be a nice turn if political reporters routinely exercised half as much critical judgment as this sports reporter does here. (And then there’s Fred Hiatt.)
Casson’s position is remarkable. How much of a scientist do you need to be to figure out that:
1. Getting participants by sending out letters to last known addresses is a terrible way to get participants. You’re automatically biasing the sample.
2. That comparing NFL players to college players rather than the population at large is a terrible way to judge the effect. It’s like determining if NBA players are tall by comparing them to college basketball players. Notice too that the college players will have played through college and in some cases, a year in the NFL. So, in my NBA analogy, you’re picking out relatively tall college players for the control group.
Also, I have a question about the approval by the Wayne State review board. My impression is that review boards do not judge the validity of a proposed design. They only concern themselves with the ethical aspects of the study – will it harm the participants, is there suitable informed consent, absence of coercion, etc. Is that correct?
It depends on the IRB and the membership. If there is a statistician on the IRB, it’s very likely that power issues will come up in the review. Designs that are biased against finding an effect are unethical generally. They are especially unethical when it is in the sponsor’s interests not to find an effect.
Will it come up in the IRB at Wayne State? That’s tough to say, without knowing the charge to the IRB (it varies some from institution to institution) and the makeup of the IRB.
Well, the NYT, New Yorker and Newsweek (and no doubt other MSM types) seem to have been put on alert. So parents, wives and perhaps even “jocks” will think twice… as long as they are able to do so.
I’d be a nice turn if political reporters routinely exercised half as much critical judgment as this sports reporter does here. (And then there’s Fred Hiatt.)
But Mark, you’re neither a scientist nor a physician, so according to to Dr. Casson, we shouldn’t listen to you.
Here’s a link to a relevant Malcolm Gladwell story that discusses this issue and more specifically, the study that is being run at UNC.
Casson’s position is remarkable. How much of a scientist do you need to be to figure out that:
1. Getting participants by sending out letters to last known addresses is a terrible way to get participants. You’re automatically biasing the sample.
2. That comparing NFL players to college players rather than the population at large is a terrible way to judge the effect. It’s like determining if NBA players are tall by comparing them to college basketball players. Notice too that the college players will have played through college and in some cases, a year in the NFL. So, in my NBA analogy, you’re picking out relatively tall college players for the control group.
Also, I have a question about the approval by the Wayne State review board. My impression is that review boards do not judge the validity of a proposed design. They only concern themselves with the ethical aspects of the study – will it harm the participants, is there suitable informed consent, absence of coercion, etc. Is that correct?
Bernard,
It depends on the IRB and the membership. If there is a statistician on the IRB, it’s very likely that power issues will come up in the review. Designs that are biased against finding an effect are unethical generally. They are especially unethical when it is in the sponsor’s interests not to find an effect.
Will it come up in the IRB at Wayne State? That’s tough to say, without knowing the charge to the IRB (it varies some from institution to institution) and the makeup of the IRB.
Dennis,
Thank you.
Well, the NYT, New Yorker and Newsweek (and no doubt other MSM types) seem to have been put on alert. So parents, wives and perhaps even “jocks” will think twice… as long as they are able to do so.