Update: I would direct interested parties to http://www.boardgamegeek.com/thread/221606 for a more rigorous and appropriate analysis of the ranking data. The model includes filters for popularity of differing degrees depending on the desire of the user and incorporate probabilistic statements of strength into the ratings interpretation.

On a couple previous Geeklists that I posted concerned with the analysis of game ratings, some fellow geeks brought up the possibility that how many times a game has been rated influences its rank (with the majority of comments leaning towards the unfairly boosts the rank of games perspective).

This Geeklist looks at the relationship between the number of ratings and the rank of games in the following manner:

1. A linear model is fit to the data (perhaps not the most accurate, but I didn't want to capitalize too much on the nature of the data). Standard linear regression was used on the ranks of both the games and the ranking in terms of the number of ratings (see the graph below)

2. The original data are detrended (sometimes called residualized). This step wil remove the influence the number of ratings has on a game's ranking, allowing the comparison between games as if they had an equal number of ratings.

3. New rankings are computed based on the residuals.

I noted this in a response below, but I will repeat this here. The List is not meant to fix all the current rankings and offer a new and perfect system. The goal is to show what happens when the #ratings is controlled for and gain comments (hopefully insightful) from the BGG community to help understand what is driving some of these ratings. This 1st attempt is based on some prior comments I received on other Geek Lists.


Conclusions: There are some definite bandwagon effects where it appears that more ratings do indeed "boost" a games rating. The correlation between the number of ratings and a games rank is a moderate .3685--indicating that the number of ratings a game receives accounts for about 13% of the information (i.e., variance) in its ranking. Removing this variance results in a different ranking that only includes the information not due to the number of ratings. Furthermore, upon looking at the biggest losses and biggest gains, they fit nicely with many user intuitions that some games are overrated (biggest losers) while some games are underrated (biggest gains).

Asumptions: The number of ratings shouldn't influence the rankings. It could be argued that better games will be rated more; however, it is convenient to assume that all games will be rated an equal number of times if we could infinitely sample and then the number of ratings would not have an effect on the overall rankings.