Glaeser on teachers

Edward Glaeser, an urban economist on the Harvard faculty widely and properly regarded as a smart guy, explains quality assurance for K-12 teaching in the Washington Monthly.  Ed, don’t risk your day job.

Glaeser’s article falls off at least two cliffs, a cautionary lesson not to opine outside the range of the data you actually have at your disposal.  First, he sounds like, um, an economist who has never really managed anything, and treats good teaching as a trait of teachers. Not a skill that can be improved and developed, a trait, like blue eyes. Teachers are what they are, so just find the bad ones and fire them.  As he is a teacher for a living, one has to assume he believes he has learned nothing about the craft since he was a wet-behind-the-ears new PhD. But just because you haven’t learned to improve your own performance is no reason to conclude no-one else can.  If, like me, he has indeed become a better teacher with coaching and practice, there’s really no explanation for not expecting the same of K-12 teachers.  And an economist should be quick to realize how much cheaper it is to train and coach than to fire and start over again and again, even if the latter worked.

He likes standardized tests, perhaps because they generate data of the usual (quantitative) type, whether or not they measure what you actually care about, and I think he wants to apply them to teachers.   Now, what would that be like?  “When John and Susie raise their hands at exactly the same time, and John is a row in front of Eddie and didn’t turn in his homework today, (A) Call on John (B) Call on Susie (C) Assign extra homework (D) Erase blackboard (E) None of the above. “  Maybe he just means external testing of students, and going back to the well of promoting teachers by score gains, and to be fair, he also wants to supplement these measures with classroom observation by experts.

But however we test whom, what’s most deaf-and-blind about this column remains Glaeser’s complete failure to say a blinking word about any way to make teachers better at what they do beyond threatening them with firing and (I guess) dangling a pay carrot.  For someone whose job is half teaching people to do stuff better than they can when they arrive, this is not just narrow-minded but incomprehensible. What does he think is happening to his students in his own courses, for Pete’s sake?  Does he think they stop learning forever when they graduate?

Author: Michael O'Hare

Professor of Public Policy at the Goldman School of Public Policy, University of California, Berkeley, Michael O'Hare was raised in New York City and trained at Harvard as an architect and structural engineer. Diverted from an honest career designing buildings by the offer of a job in which he could think about anything he wanted to and spend his time with very smart and curious young people, he fell among economists and such like, and continues to benefit from their generosity with on-the-job social science training. He has followed the process and principles of design into "nonphysical environments" such as production processes in organizations, regulation, and information management and published a variety of research in environmental policy, government policy towards the arts, and management, with special interests in energy, facility siting, information and perceptions in public choice and work environments, and policy design. His current research is focused on transportation biofuels and their effects on global land use, food security, and international trade; regulatory policy in the face of scientific uncertainty; and, after a three-decade hiatus, on NIMBY conflicts afflicting high speed rail right-of-way and nuclear waste disposal sites. He is also a regular writer on pedagogy, especially teaching in professional education, and co-edited the "Curriculum and Case Notes" section of the Journal of Policy Analysis and Management. Between faculty appointments at the MIT Department of Urban Studies and Planning and the John F. Kennedy School of Government at Harvard, he was director of policy analysis at the Massachusetts Executive Office of Environmental Affairs. He has had visiting appointments at Università Bocconi in Milan and the National University of Singapore and teaches regularly in the Goldman School's executive (mid-career) programs. At GSPP, O'Hare has taught a studio course in Program and Policy Design, Arts and Cultural Policy, Public Management, the pedagogy course for graduate student instructors, Quantitative Methods, Environmental Policy, and the introduction to public policy for its undergraduate minor, which he supervises. Generally, he considers himself the school's resident expert in any subject in which there is no such thing as real expertise (a recent project concerned the governance and design of California county fairs), but is secure in the distinction of being the only faculty member with a metal lathe in his basement and a 4×5 Ebony view camera. At the moment, he would rather be making something with his hands than writing this blurb.

25 thoughts on “Glaeser on teachers”

  1. Read his papers and Triumph of the city. Interesting urban economics and policy guy and near sainthood for TOD advocates.
    I guess if I went to Collegiate (like Ed), Trinity, or Horace Mann,
    instead of Stuy or Bronx Science (which ironically admission is based on test scores)
    and went Ivy all the way, I would have a platform to discuss GE Neutron Jack’s tactics for education and test scores.

    Aside from the ad hominem, the paper reads like a page from corporate America of “Up or Out” with a mixture of Six Sigma Black Belt
    tactics, which I was regretfully involved in my past life. I am sadden that this soul crushing methodology has somehow
    percolated into academia.

    1. Hmm. I tried to like Triumph of the City. I will have to give it another try in the future. The way it was laid out, I couldn’t see where the footnotes were, and without those, I have trouble taking someone seriously. But since I am someone who tends to disagree with most of these “urban economists,” I should try it again I guess.

      1. I’ve studied his work and took UrbEcon in grad school. I think the discipline is OK as far as it goes. If you look at the r^2s and keep in mind that it explains some of the variance, but doesn’t have compelling explanatory power, you’ll be OK. I look at the work and think ‘that’s kind of interesting’ and move on. It is only a minor part of informing policy, not a major part. But a part.

        1. Agreed. It seems intuitive that as social creatures, we need to be together, often, to do our best work. I just don’t see how one gets from that to the current interpretations of TOD, which seem trendy to me. Anyhoo.

  2. “If, like me, he has indeed become a better teacher with coaching and practice, there’s really no explanation for not expecting the same of K-12 teachers.”

    Maybe he’s concluded that there’s a significant fraction of K-12 teachers who really are in the wrong line of work, and won’t benefit from coaching and practice?

      1. You’d have to ask him, not me, I was just suggesting what sort of reasoning he might have been engaged in.

  3. OOO! I know the answer to the question about which student to call on! Pick me!

    An anecdote about my kid’s fifth grade math teacher: Mrs. R used the ice cream stick method to choose students to call on. The way this method works is, there are two jars to start off with, one empty and one full of ice cream sticks, one stick for each student with their name on the stick. The teacher pulls out a stick and whoever’s name is on the stick, has to answer the question. That stick is then placed in the empty jar. When all the sticks are finally in the second jar, the teacher starts all over again.

    The reason educators came up with this trick is that there is a body of research that shows that subtle prejudices influence who gets called on, and how frequently. This random method corrects for that.

    I only learned about the ice cream stick method during our parent-teacher conference. Mrs. R told us that she sometimes held her breath when our son’s name was on the stick and the question was a difficult one, fearing that he wouldn’t be able to answer (he has autism), but to her delight, he almost always was able to answer the question correctly. Even when it looked like he wasn’t paying attention, he was.

    What’s the point of this long story? Public school teachers take on everybody, from the non-verbal, intellectually disabled kindergartener on up. It’s an incredibly complex undertaking. What Glaeser does in his classroom is simple in comparison. Relatively few people go to college and even fewer go to Harvard.

    If Glaeser has weaknesses as a teacher, the young adults sitting in front of him are able to find ways to privately compensate for them and succeed in spite of his deficiencies. If all he gave were multiple choice tests, it wouldn’t matter, his students have already honed their composition skills elsewhere.

    If my kid’s teachers concentrate solely on the skills needed to do well on multi-choice tests, he will not learn to write well. And I’m sorry to say, that thanks to the likes of Glaeser, my kid DOES spend a disproportionate amount of time being taught to the test and not enough time on composition skills.

    1. “Public school teachers take on everybody”
      How true! The best teacher I ever had was my electronics shop teacher: Mr. Shultz. He had one of the hardest classes imaginable to teach, because about half the students were vo-ed, and the other half were science nerds (days before personal computers). The vo-ed types were serious students, but the science nerds had most of the candlepower. I never knew how he did it, but he managed to keep both groups engaged at the same time. Well, lab worked helped–it was a bit of an equalizer. But he somehow ran the lecture part at two levels at the same time.

      1. That wouldn’t by any chance have been at Almont High School, would it? I had a Mr. Shultz as shop teacher there.

    2. Ohio Mom, that’s a good answer, but it’s not the official right answer on the rubric they gave us, so I’m giving you a zero for this question. Please stop reading stuff that wasn’t assigned and distracting the class with it.

  4. There is indeed too much emphasis on standardized testing. But there is also a population of teachers who are so ineffective that coaching is not the answer, either because they don’t have the ability to improve or they resist improvement (and it’s true that for years there was no need to improve once they had tenure). The problem, as has been pointed out elsewhere, is that there are not necessarily any good replacements available if ineffective teachers are terminated.

  5. Teachers are of marginal importance. Socio-economics is everything. Kids have profoundly different experiences from birth, owing largely to structural perpetuation of class structures. Aside from taking *that* on directly, via larger social and economic transformation, we can redesign schools just as profoundly so as to correct for these disadvantages. Until we at least do the latter, I really don’t want to hear about fiddling with test scores, evaluations, curriculum, incentives, charters, etc.

    1. Neil Postman used to say, most notably in The End of Education that given a sufficiently strong ‘why’ to learn, students would survive, even flourish, under almost any ‘how’.

      We can’t claim to be training them for jobs, and the robots will finish off that rationale.
      Preparing students for responsible citizenship is, unless you own a casino in Macau, seems to be losing its salience.
      We’re not providing them with the protective coloration needed for social mobility any more, because of what’s happened to social mobility.
      Delivering to them the residuum of their culture, and preparing them to transmit it to future generations is an aim as shattered as the notion of a single ‘culture’.

      Postman said, “without a transcendent and honorable purpose schooling must reach its finish, and the sooner we are done with it, the better.” Which is why in the first grade students run off the bus to get into the building, and in the tenth they run out of the building to get on the bus.

  6. Ed Glaeser is a distinguished tenured professor at Harvard. From this, Michael seems to infer that he has spent some time as a teacher. Knowing a little something about undergrad education at Harvard, I can’t see the connection.

    1. Michael is correct if he infers Glaeser spent some time as a teacher.

      Unfortunately, (a) it was a long time ago, and (b) his experience dealing with 99th percentile students doesn’t accurately portray the problems of public school teachers.

  7. Someone, anyone, please help me out. There has been a lot of talk, criticism actually, out in the wilderness, and, right here at RBC, about, “teaching to the test, “ and/or, “way too much ’standardized’ testing.” What, exactly, does that mean? Unless you have the actual answers to specific tests, and give them to the students, how, exactly, do you “teach to the test?”

    Or, is this a not-so-veiled criticism of testing in general? And, if so, how does one suggest a teacher gain the knowledge of what students have learned, or not?

    1. From past experience you know what sort of questions are on the test. There is a list of required topics, for example. You teach nothing but the topics that will be on the test. You drill them over and over and over. And your kids don’t learn any of the many things not on the test.

      Oh, and you even can start teaching to specific types of questions – which moves away from teaching kids to write, or read, or do math in general. It’s incredibly destructive to actual learning; the kids get to hate the repetitive drills and the subject matter. You also don’t get to learn the general rules that give you actual mastery, instead memorizing disconnected bits.

      1. I think this attack on “teaching the test” is a bit exaggerated. The thing about teaching the test is that the students at least learn the test, and if the test is well constructed, it’s real learning.

        You can’t teach the math test without teaching math, you can’t teach the chemistry test without teaching chemistry. And, sure, if the English test is about Anne of Green Gables, you’re not going to be teaching The Rime of the Ancient Mariner if you teach the test, but they’ll at least learn Anne of Green Gables.

        It might not be ideal, IOW, but it sets a floor.

        1. The thing is, though, they aren’t well-constructed. Virtually ever. They are way too low-level for kids who are smarter but still a lot of effort for kids who struggle. Since tracking doesn’t start in most places until the kids are older, you waste the time of the kids who are bright, which makes them disengaged and lazy. To note just one of many problems.

          1. But you’re not describing a problem with teaching to the test, you’re describing a problem with late initiation of tracking. ANY lesson plan is, tautologically, going to be difficult for students who struggle, and any lesson plan that’s ok for the majority of the class is, without tracking and AP going to be too easy for the advanced students.

            The problem here is with grouping students by age, and not accomplishment level. Not with teaching methodology, unless you’re expecting the teacher to, in effect, simultaneously provide individualized instruction to a dozen or two students.

            My wife, an elementary school teacher from the Philippines, says that there they simply advance the students as fast as they learn the material, irregardless of age, so that a student might be with age mates in history, and children a couple years older in math. But they’re a developing nation, things like that are too complex for the first world to practice.

    2. in texas, where i teach, teaching to the test means being given a curriculum document called the teks (texas essential knowledge and skills) which provides a list of things that are supposed to be learned in each subject in each grade. math and reading are tested every year from third grade through high school. other subjects are tested at particular grades (science, for example, is tested at the end of fifth and eighth grade while writing is tested in fourth and seventh grades)and the high school grade level tests are being replaced with end of course exams. a selection of those things that are to be taught are picked by the state for the actual tests. the test scores are provided to the students, parents, and schools but the tests themselves are not revealed every year so that the teachers mostly have no idea what the kids are actually having trouble with. every five to seven years or so texas teachers manage to adapt themselves to teaching the curriculum successfully enough so that most students in the state can pass the test and so every five to seven years or so the texas legislature or the state education bureaucracy declares that the current test is no better than a literacy test and replace it with a testing regime that is more “rigorous” at which point most students can no longer pass the test. i’ve seen this cycle play out 5 times over the past 30 years. i expect to see it play out another time or two before i retire.

      here’s a link to the state curriculum for mathematics in the middle school grades–

      http://ritter.tea.state.tx.us/rules/tac/chapter111/ch111b.html

    3. The general problem (which exists to an extent outside of testing, but can be exacerbated by it) is when you are teaching retention rather than (transferable) mastery of a subject. Retention allows you to reproduce knowledge and apply skills in a predictable fashion; mastery allows you to apply skills even outside the situations that you had learned. Another related problem, is the emphasis on summative assessment over formative assessment.

      Test structure plays a role, too: It is very difficult (though not impossible) to test mastery using multiple choice tests (even the subject GRE is very heavy on retention). It’s difficult to weigh questions accordingly, and multiple choice allows you to take shortcuts that you cannot take when you have to expressly derive the solution. This does not mean that free response questions are automatically mastery- rather than retention-based; they often aren’t; they are necessary, not sufficient for testing mastery. Similarly, multiple choice tests in particular (and many standardized tests in general) are predominantly summative in nature and have limited usefulness for formative purposes.

      The performance of United States students on TIMSS and the PISA math component illustrates this problem and that it is very likely real and not just a talking point of NCLB critics: Our students do well on TIMSS, but not so well on PISA math. The difference between TIMSS and PISA is that TIMSS is curriculum-based, while PISA focuses “on young people’s ability to use their knowledge and skills to meet real-life challenges, rather than merely on the extent to which they have mastered a specific school curriculum.” As a further complication, the curriculum that TIMSS is based on matches US curricula well, but not necessarily those of other countries.

      I often struggle to explain how different my experience at a German Gymnasium was compared to my American highschool in this regard [1]. I did not see a single multiple choice test during my time in Germany; instead, tests tended to have liberal helpings of what they called “Transferaufgaben” (transfer tasks). A transfer task requires students to transfer their knowledge to a new problem in ways that they haven’t done yet, or to combine it with a different piece of knowledge in new ways. An example from a German matriculation exam (advanced mathematics) was the following: Considering a family of functions f_k: x->(x^2+1-k)*e^(-x) for real-valued k, which of them have a horizontal tangent? This requires the following steps: Figuring out that you need to solve f’_k(x) = 0 for x; apply the product rule to find the formal derivative; find that f’_k consists of two factors, e^(-x) and a quadratic polynomial; know that for f’_k to have a root, the quadratic term must be zero, because e^(-x) can never be zero; and solve the quadratic equation.

      My German teachers considered transfer tasks essential for teaching; they challenged students in way that testing for retention could not and required us to acquire an understanding of the subject matter that was deeper than just being able to reproduce what we had been told. They also provided guidance in that they gave them a clearer idea of where there class (or indivdual students) was struggling (i.e. formative vs. summative assessment). Of course, transfer tasks are problematic for standardized tests; they rely on students having the correct insight aside from having the necessary basic knowledge, and thus produce results with a higher variance.

      Note that this does not mean that such a test is necessarily harder: In my experience, American high school tests could be just as challenging, but for different reasons: They usually required answering a lot of questions in a limited time, so quick comprehension and having the requisite knowledge (often a lot of knowledge) at your fingertips was essential.

      I’ve put translations of some Bavarian matriculation exam questions from 2005 up here (both translated from their originals at the Spiegel page) to illustrate the typical style. Both are basic, not advanced exams, though passing a matriculation exam certifies you for college admission, so the difficulty level is closer to our AP exams than normal high school exams. I’m putting them up mostly to illustrate the style: Total absence of multiple choice questions; deriving a solution is often more important than providing it (in fact, some intermediate results are provided as part of the calculus test so that students don’t get stuck if they can’t deal with some of them); the history questions are very open-ended; students have approximately 105 minutes to answer the five questions (they have two answer two such sets of questions and have 3.5 hours for all of them), so expect to do a lot of writing.

      [1] Let me add that this positive experience is not universal in the German system, which is why I’d advise against using the system as as a model; in particular, the “Hauptschule” track shares many of the problems of our poorer schools, in that it easily can become a dumping ground for low SES and struggling students. I’m talking here solely about test design in those schools that do work and how that relates to teaching and learning.

    4. You ask, what is a test-prep class? I have some first-hand experience to share.

      Over the years, my kid has bounced between language arts in the general ed classroom (what some of you most likely think of the “regular” classroom) and classes that primarily focus on test prep. He was in test prep classes in fourth, fifth and seventh grade; seventh grade was the year we relented and hired a special ed professor to tutor him in LArts so he could catch up from all he’d missed in the test prep classes.

      Anyway, for an example of what goes on in a test prep class, let me outline what was done in fourth grade, when the euphemism for this class was “pull-out.” My kid and a few others left their class during the language arts lesson and met with the special ed teacher in her office.

      For writing instruction, the main thing they did was complete a workbook, which contained sentences and paragraphs with punctuation and grammar mistakes that they were to identify and correct. This allowed them to ace similar “writing” questions on the test. (What? You don’t think that qualifies as writing?)

      For reading they read what are called “contrived” stories, stories that are written to be part of reading tests, and answered mostly “look back” questions. A look-back question is just what it sounds like: “Where was Little Red Riding Hood going?” The answer takes no analysis, just re-reading.

      They did not read the same books their peers were reading back in the general ed classroom, they did not do book reports, they did not write personal essays, they did not write what passes for a research paper in elementary school.

      They did next to no original writing, never read anything more than two pages long or by an actual author, and yet they had the skills to pass the standardized test.

      For another example, in the seventh grade “standards class” (different school building and level, different euphemism), many Fridays were devoted to specific test-taking skills, such as practicing the process of elimination for multiple guess question. It was a signficant chunk of istructional time.

      You also ask, how do teachers gain the knowledge of what their students have learned?

      Putting aside the question that not everything can be assessed on a standardized test (e.g., public speaking skills), experienced teachers know their students
      and are also quite capable of making up their own exams.

      There is a place and a role for some standardized testing but it is nowhere near the size we are giving it now. I’ll add that this year my child is taking three separate sets of MAP tests (at the beginning, middle and end of the year) and one week worth of practice graduation tests (one for each subject area).

      The MAP tests do not align with the curriculum and the graduation tests are supposed to be changed before he is old enough to take them for real. How can this all be a good use of his and his teacher’s time? The only information his teachers are getting from these tests is whether or not he’ll be able to do well on the next set of tests!

Comments are closed.