Toward Crowdsourced Behavioral Ratings of Movie Quality

Last night I tried to watch an arty, independent film streamed over Amazon. It was poorly constructed and had many slow spots, so I paused it to play a few hands of hearts on the computer, then did so again later to make a phone call, then again later to fold some laundry. I never scrolled back to re-watch any scenes because there was nothing that I really wanted to see twice.

I will not write one of my weekend film recommendations for the movie because it wasn’t good enough to be worth the time. Yet a good deal of data are available for me and everyone else who has streamed the movie.

I would be interested to see a streaming site have all viewers consent to public reporting (in the aggregate I mean, not person by person) of their passively gathered data on films, e.g., how many people who started the movie stopped watching and never finished, how many had long pauses, how many re-watched etc. These are crude data relative to film reviews, but on the other hand are more representative of the audience.

You can never tell how many films a critic watched before s/he chose to write a review of one in particular, nor can you know whether the critic is a buddy of someone in the movie and is therefore over-praising it when in fact the typical viewer will hate it. Anyone can write a lavish review of their pal’s movie (or their own) on Amazon, but they couldn’t fake the reaction of say, 50,000 viewers who watched it without stopping, 10,000 of whom watched it again within the same week because they so enjoyed it.

Author: Keith Humphreys

Keith Humphreys is the Esther Ting Memorial Professor of Psychiatry at Stanford University and an Honorary Professor of Psychiatry at Kings College Lonon. His research, teaching and writing have focused on addictive disorders, self-help organizations (e.g., breast cancer support groups, Alcoholics Anonymous), evaluation research methods, and public policy related to health care, mental illness, veterans, drugs, crime and correctional systems. Professor Humphreys' over 300 scholarly articles, monographs and books have been cited over ten thousand times by scientific colleagues. He is a regular contributor to Washington Post and has also written for the New York Times, Wall Street Journal, Washington Monthly, San Francisco Chronicle, The Guardian (UK), The Telegraph (UK), Times Higher Education (UK), Crossbow (UK) and other media outlets.

3 thoughts on “Toward Crowdsourced Behavioral Ratings of Movie Quality”

  1. Good luck getting the big studios to agree to allow the major streaming sites to make that data publicly available, for the big-budget or -hype movies at least.

  2. Those data are too valuable to share:


    “Netflix doesn’t know merely what we’re watching, but when, where and with what kind of device we’re watching. It keeps a record of every time we pause the action — or rewind, or fast-forward — and how many of us abandon a show entirely after watching for a few minutes. …

    For at least a year, Netflix has been explicit about its plans to exploit its Big Data capabilities to influence its programming choices. “House of Cards” is one of the first major test cases of this Big Data-driven creative strategy. For almost a year, Netflix executives have told us that their detailed knowledge of Netflix subscriber viewing preferences clinched their decision to license a remake of the popular and critically well regarded 1990 BBC miniseries. Netflix’s data indicated that the same subscribers who loved the original BBC production also gobbled down movies starring Kevin Spacey or directed by David Fincher. Therefore, concluded Netflix executives, a remake of the BBC drama with Spacey and Fincher attached was a no-brainer, to the point that the company committed $100 million for two 13-episode seasons.”

  3. Interesting. There’s an inherent bias, since the audience for streamed movies is not the audience for movies, but still the data would be useful.

    Others are probably less concerned with the way new TV technology permits the TV to watch the audience instead of the other way around. But this makes me less likely to watch streamed content. Do I really want some corporation to know how many times I rewatched the sex scene?

Comments are closed.