More classroom flipping: testing as learning

The “flipped classroom” pedagogical model is the hot ‘new’ idea in my industry. The idea is to move didactic delivery of knowledge (especially facts) out of the classroom to venues better suited to it, like the web and books, and use live meetings of students and prof for coached use of that knowledge in discussion, exercises, and the like. New, as in, “this is how art, mechanic arts, and sports have been taught for thousands of years”.   I have pretty much drunk this Kool-Aid, partly on the evidence that active learning is the universal practice everywhere the task is to acquire a skill, including very high-level skills. No-one begins teaching the piano with a reading assignment or a lecture.

Another way to “flip” the learning experience is to move testing from a big-deal end-of-the-course high-stakes assessment to frequent, low- or no-stakes events throughout the course. There’s a lot to like about this on its face, starting with the near-complete dissimilarity between what conventional exams measure and what education should be making people better at. I also dislike the infantilizing affective tone of exams that can be graded with an answer key: grownups who quiz each other as a social convention “Hey, guess what I read in the paper this morning.  No, really, guess!” don’t have a lot of friends.

It turns out that low- or no-stakes testing right after we get a dollop of knowledge helps us retain the knowledge, and better than restudying or reviewing the material.  This seems to be the key paper , and why haven’t I known about this for eight years?  Active learning of course has a lot of this built in: after you play a passage on the piano and get some coaching on what you did, you actively ‘recall’ what you learned by playing it again, or playing something else.  It seems that for enduring recall, the task of retrieving something from memory is as important to improve with practice as the act of committing to memory.

The effect size here is notable, and look at those error bars:


Remarkably, you don’t even have to actually ‘take the test’ in the sense of writing down your answers.  And it seems that learning your score on these tests doesn’t matter, never mind whether they ‘count for your grade’.

I think, on the strength of this research, I have to make space in every class session for quick testing , and figure out how to design such tests for the kind of material I teach. Students are mistaken about this stuff, and incorrectly predict that they will remember material better after a week when they study it repeatedly than when they study it once and are tested on it. (Of course, students are mistaken (or misinformed) about a lot of learning technique, like highlighting (lose it).) They will grouse, but at least I have some real research to reassure them with.

Author: Michael O'Hare

Professor of Public Policy at the Goldman School of Public Policy, University of California, Berkeley, Michael O'Hare was raised in New York City and trained at Harvard as an architect and structural engineer. Diverted from an honest career designing buildings by the offer of a job in which he could think about anything he wanted to and spend his time with very smart and curious young people, he fell among economists and such like, and continues to benefit from their generosity with on-the-job social science training. He has followed the process and principles of design into "nonphysical environments" such as production processes in organizations, regulation, and information management and published a variety of research in environmental policy, government policy towards the arts, and management, with special interests in energy, facility siting, information and perceptions in public choice and work environments, and policy design. His current research is focused on transportation biofuels and their effects on global land use, food security, and international trade; regulatory policy in the face of scientific uncertainty; and, after a three-decade hiatus, on NIMBY conflicts afflicting high speed rail right-of-way and nuclear waste disposal sites. He is also a regular writer on pedagogy, especially teaching in professional education, and co-edited the "Curriculum and Case Notes" section of the Journal of Policy Analysis and Management. Between faculty appointments at the MIT Department of Urban Studies and Planning and the John F. Kennedy School of Government at Harvard, he was director of policy analysis at the Massachusetts Executive Office of Environmental Affairs. He has had visiting appointments at Università Bocconi in Milan and the National University of Singapore and teaches regularly in the Goldman School's executive (mid-career) programs. At GSPP, O'Hare has taught a studio course in Program and Policy Design, Arts and Cultural Policy, Public Management, the pedagogy course for graduate student instructors, Quantitative Methods, Environmental Policy, and the introduction to public policy for its undergraduate minor, which he supervises. Generally, he considers himself the school's resident expert in any subject in which there is no such thing as real expertise (a recent project concerned the governance and design of California county fairs), but is secure in the distinction of being the only faculty member with a metal lathe in his basement and a 4×5 Ebony view camera. At the moment, he would rather be making something with his hands than writing this blurb.

One thought on “More classroom flipping: testing as learning”

  1. I understand the French Foreign Legion run a language course for new recruits who aren't native speakers like (Walloon) Belgians. The pass rate is very high, though the students are not very like Mike's (or at least for his sake I sincerely hope not). Nor is the vocabulary exactly standard: the coachman's aunt has just been hit by an RPG fired from behind the building 10 o'clock from the bushy-topped tree, fragmentation grenades, 5-second fuse, fire. I assume it works by motivation (pass or out) and drill instructor feedback.

Comments are closed.