How to Rate on a Scale of 1 to 10

Alright, I’ve been getting some criticism for my reviews, some people pointing out that other people say it’s good, so it’s good. While I don’t wish to negate the input of these other players (that rarely tell you ‘what’ exactly is good) I do want to point out that there’s a good chance that they’re reviewing your game incorrectly. So let’s talk about the 1-10 scale.

I don’t think I’ve ever seen a scale utilized as poorly as 1-10. The reviewer is offered 10 options to choose from, which may be a bit cumbersome. So cumbersome in fact, that I think people completely disregard some of the scale to make it less cumbersome. If you happen to be a human reading this above the age of… let’s say 15 years-old, I want you to think of all of the people that you’ve rated based on attractiveness. More specifically, the ones you rated on a scale of 1-10. Ladies, you play along too, I know you rate guys. Now I want you to think of the distribution of those ratings. You probably have a lot of 7’s, 8’s, maybe a few 6’s and 3’s, with very few 1’s and 10’s. Those of you familiar with the Bell Curve will no doubt see the problem here, but I’ll explain it for those that don’t.

This is the general shape of the Bell Curve. This one in particular is a normalized distribution, so it’s designed to show probabilities of an event occurring. And what we see here is that the greatest number of events should be rated 6. This is because 6 is the average between 1 and 10 (Actually 5.5 is but we’re rating using whole numbers so round up or round down. I choose to follow convention and round up). As the ratings deviate from the mean, we should see fewer and fewer uses of these ratings. So there should be more 3’s than 1’s and more 8’s than 10’s. If you’ve rated 100 people based on appearance, you should have very few 10’s. But I’m willing to bet that you have more than the appropriate number of 10’s and not enough 5’s or 6’s. Why is that? Let’s continue

The normal person will likely have a lot of 7’s in their rating database. I believe that when one asks themselves how attractive someone is on a scale of 1-10, they’ve unconsciously set 7 to be the mean. Movie was average? It’s a 6 or a 7. Since this close to the deviation from the mean, it actually produces okay results in ratings. However when it comes to terrible movies, it seems no one knows what to do. Is it a 3? Is it a 4? You’ll likely find that there are more 3’s and 4’s than there are 5’s, despite 5 being closer to the mean. And probably more 8’s than the two combined despite being aligned with 4 on the distribution.

Are you just seeing more terrible movies than slightly worse than average movies? Are you seeing more great movies than more terrible movies combined? That’s a distinct possibility. After all, who willingly watches a movie that they expect to be terrible? Who willingly plays a game that they expect to be terrible? But in that case, we should see a slightly shifted curve, rather than, well, a non-Bell Curve. If you look at seasonal anime ratings on MyAnimeList, you will see many 7’s and not nearly enough 5’s or 6’s. This is evidence of a shifted mean.

I’m not judging anyone for this behavior. I used to engage in this myself, particularly regarding anime and manga. But once I sat down and asked myself why I rated SAO a 6 and several of the Monogatari series in a similar range (5-7) I realized the problem. I thought back to the humans that I had rated based on appearance and saw a similar trend.

Bringing this back to what the problems with ratings, people seem to exclusively use a 3-10 scale instead of a 1-10 scale. The removal of 2 ratings might not seem significant but you’re talking about 20% of your rating scale not being used much at all. And then we need to remember that the lower ratings 3-5 are not used much at all. So you’ve pretty much turned the rating system into 6-10 with an average around 7, which actually lines up with general public nicely. If we turn it into a 1-4 system and equate 7 with 2, then 50% becomes the mean and that’s about what we want.

So I guess it’s not that you’re not producing a Bell Curve properly, it’s that you’re producing it for the wrong range of numbers. If you want to use a 1-10 scale, you need to USE the full scale. You can’t toss out 9’s and 10’s like candy because you felt something was phenomenal. You need to think about all of the games you’ve played up until now and see if it’s not really an 8. And don’t forget about the lower numbers. Don’t just hate a game or a movie and say, “Yep, that’s a 3”. Think about what you didn’t like and compare it against all of the others you’ve seen before.

At the end of the day, the method is up to you, but by adhering a bit more strongly to a 1-10 scale, you can make your ratings on the 1-10 scale be a bit more meaningful. I want to encourage you to really use the 1-10 scale and not the 6-10 scale. Anyway, thanks for reading.

Artemis Hunt