Quantified – Top 10 Horror Films of All Time. Really?

Halloween has come and gone, and hopefully those of you with an affinity towards films that like to scare the bejeezus out of you, you were able to watch enough horror to get you through another year. Of course, most of us horror fans only use Halloween as an excuse to get more of the terrifying treat than normal, which is how I justified the bad timing of this article. Can you tell I’m not a professional blogger? I have a post on horror movies almost two weeks after Halloween.

But I read something recently that I couldn’t stop thinking about. The article I read is from IndieWire and is titled IMDb Users List Top 10 Horror Movies of All Time, From ‘The Shining’ to ‘Sweeney Todd’. Let that sink in. Go ahead and read that title one more time. The Shining and Sweeney Todd were just compared to one another as all-time horror greats. I didn’t realize Sweeney Todd was a good movie, let alone a horror film. So I did a little digging into the origin of this list (if you can count a google search as ‘digging’). I found the original post at IMDB, which is a little anti-climactic. It doesn’t tell us really anything about how the list was created. What we get is this:

 Here are the Top 10 Horror Movies of All Time, as determined by user votes and ratings. Do you agree with the films?

I can’t think of any horror aficionado who would agree with this list. I’m assuming this list was put together with the sole intent of causing disagreement. According to IMDB, some criteria was used to create this list, but that sure doesn’t clear things up. User votes and ratings. Is that implied as both user votes and user ratings? What’s the difference between a vote and a rating? And that doesn’t begin to explain the casual genre classifications. Here’s the list of films along with the average IMDB user rating (votes?).

Ranking	Film Title	IMDB User Score
1	The Shining	8.5
2	Alien	8.5
3	Shaun of the Dead	8
4	Psycho	8.6
5	Cloverfield	7.2
6	Zombieland	7.8
7	Saw	7.7
8	The Exorcist	8.1
9	28 Days Later	7.6
10	Sweeney Todd	7.5

The table I created will allow you to sort. Go ahead and sort it by IMDB score. Notice anything? Clearly the order has shifted, which makes you wonder even more about what IMDB is doing to get this list. But the thing that stands out to me is which films are now on the bottom of the list. Those are the films that most people would agree do not belong on this list. Shaun of the Dead is a great comedy about zombies. It is not a horror film. Cloverfield, which is a movie I like, is in no way a top ten horror film. Saw, which also fits the genre, just doesn’t have the chops to compete with so many other great horror options. Zombieland is in the same boat as Shaun of the Dead, 28 Days Later, another horror film I like, could be argued for inclusion, but I think it would lose more often than not, and then there’s Sweeney Todd. There is only one film Tim Burton has directed that I dislike more than Sweeney Todd, but let’s chalk that up to personal opinion that I know may not be echoed commonly. Still, Sweeney Todd is in no way a horror film. What is going on with this list?

If IMDB was using user submitted scores to rank these films, why wouldn’t they be in that order? Where is Rosemary’s Baby (8.0), The Thing (8.2), or even The Silence of the Lambs (8.7) since we’re being generous with genre classification? Each of those three films outweighs any of the dubious inclusions listed above. My suspicion here is that IMDB is using user scores over time. IMDB does a great job of showing the details of explaining how scores are distributed. At least to a point.

If you follow the link next to user ratings on any movie profile page on IMDB, you will be taken to a page that shows distribution data for each choice, in IMDB’s case, 1 – 10. You can also get a user rating report by gender, age, and IMDB staff, international users, and Top 1000 users. If IMDB is generating this horror list by users who voted since last Halloween, for example, there is no way for me to track that. It would be great to see votes submitted by month or year, but I don’t know of any way to get that information. I’m sure it’s possible on their side of things, but it isn’t something that’s available to the likes of me.

But could there be a way to simulate some understanding of votes over time? If you allow me to take some liberties, I think it might just be possible. At least for exploration purposes. Age and time are related. IMDB sorts votes by age groups. It’s not clear to me whether the age group is tied to a user account or if it’s tied to a specific vote (for example, if I submitted a vote for a film while I was 17, when I turned 18, would that vote still be counted in the under 18 group or does it travel with me as I age? I don’t know the answer to that, but I think looking at the age groups will help us get an idea of what might be happening with this list. Since the movies on this list that seem less deserving are newer films, it’s also possible that age group is favoring newer films and if the votes are over time, these films would rise to the top. Just for shits and giggles, let’s look at the age distribution of scores for two films, The Shining and Cloverfield.

The Shining IMDB Score by Age Groups

Age Group	IMDB Score
Aged under 18	8.7
Aged 18-29	8.6
Aged 30-44	8.5
Aged 45+	8.2

This looks pretty normally distributed, right? Of course all of these films should be pretty closely distributed based on their average scores already being pretty high and the number of votes being relatively large. For comparison now, let’s look at Cloverfield.

Cloverfield IMDB Score by Age Groups

Age Group	IMDB Score
Aged under 18	7.4
Aged 18-29	7.4
Aged 30-44	7
Aged 45+	6.6

Clearly the distribution isn’t as tight with Cloverfield. Younger audiences liked it more than the old folks. This isn’t shocking news, horror is a younger audience’s game, but where there is an obvious difference with Cloverfield and age groups, we don’t see quite the same difference with a film like The Shining. Sure, The Shining is older, and that’s something to keep in mind, but it’s not enough to explain the difference, in my opinion.

If we take the age groups from these two films and find the standard deviation, we can get a numeric value for understanding the slight changes in score between al l the age groups. Standard deviation is often used for more advanced purposes, but for us, just being able to see the distribution of the averages is enough to do some comparing. So how do The Shining and Cloverfield stack up?

The Shining Versus Cloverfield in Age Group IMDB Scores

The Shining	Cloverfield
0.187082869 Standard Deviations	0.331662479 Standard Deviations

The lower the deviation the more closely distributed the range is. The two numbers are close, depending on your idea of close and depending on what we’re comparing. Here, we’re comparing two numbers that are already very close, so small differences can be important. For Cloverfield, the average score fluctuates up or down by a third of a point by age group. The Shining changes less than 2/10ths. The real fun begins when we do this for all the films and chart the results from lowest standard deviation to the highest. Here’s what it looks like:

What we can see here is further evidence that those dubious looking films, the ones any horror film fan would wonder how the hell they got on a best ever horror film list have the highest standard deviation, suggesting a disagreement between age groups on the quality of the film. For Sweeney Todd, there’s half a point difference, which is a large spread given that these films are supposed to represent the cream of the crop. It’s also suggestive that this list is perhaps the best horror films for a new generation. But I don’t buy that. In fact, I think the problem is more likely to do with exposure than taste. That is evidenced by there being a low standard deviation among age groups for the older films.

If we sort the films by the youngest age group (how young are these kids, anyway?) the list isn’t too far from how the older age groups rank the films. Sure, The Exorcist is too low, but the youngsters just rated Psycho as the best, and who would really want to argue that?

IMDB's Horror List Sorted by Under 18 Scores

Film Title	Score by Users Under Age 18
Psycho	9
The Shining	8.7
Alien	8.6
Shaun of the Dead	8.3
Zombieland	8.3
The Exorcist	8.2
Sweeney Todd	8.2
28 Days Later	7.9
Saw	7.9
Cloverfield	7.4

Has this restored your faith a little? It did for me. Except for Sweeney Todd. And except for the genre abusing, but again, that’s out of the scope for this post. At least for now.

But I wanted to add one more thing. I mentioned before about including three films, Rosemary’s Baby, The Thing, and The Silence of the Lambs and I wanted to see where they would fall on the standard deviation line chart. So I’ve added those three. Again, the genre for The Silence of the Lambs may not be true horror, but the list is already somewhat bastardized in that area and that film has been included on other best horror lists, so at least I’m not the first. There are other films that could easily be slipped into the group, but I chose these three because they were the first ones to come to me. It’s that simple.

Unless IMDB came out and told us how it created that list, we will never know for sure. And since I can’t check user votes over time, we’re limited in the analysis, but I think we can draw a few conclusions from the short analysis I’ve included here. First, age groups play a role, and if the votes are counted over time, this is likely why the list has appeared the way it has. Second, it’s not really an issue of the younger viewers not appreciating old movies, as might often be touted by older viewers. With the younger audiences choosing Psycho as the best, the issue is probably more likely exposure to older films. The number of votes by age is of course very low with the younger viewers, as it probably should be for a number of reasons. But given the chance, old films stand up very well with newer audiences.

The larger issue here, I think, is in genre classification. Is Alien an actual horror film? How much comedy in a horror film is allowed before it’s a comedy with horror elements? Can a musical, even one rated R, ever really be a true horror film? Maybe there are more questions raised here than answers, but at least I feel confident that some horror classics like Psycho, Rosemary’s Baby, The Shining, and The Thing can still impress young, new audiences.