Oppressed by Pessimists / Fueled by Optimists >> Games with personalities? (v2)
Joe Grundy
Australia
Sydney
NSW
-
Oops. I published this earlier but my underlying number crunching had a mistake, which of course I've now fixed.
This is an exercise in statistical curiosity... some game ratings might be dominated by a bunch of pessimists dragging it down, or a bunch of optimists giving it a boost.
There are 25,000 game raters on BGG and 3,500 ranked games. An average ranked game has 260 raters, but that's mostly because a few games have thousands. Half of ranked games have fewer than 90 raters. What if those 90 (or fewer) happen to be characteristically low or high raters, either by chance or because the game attracts mostly negative b*stards to play it?
So I thought I'd check it out. Here's the waffle, followed by the list of most affected games...
First I assigned each user a User Optimism value. This is the average value of their own rating for each game above the game's overall BGG rating. ie how much higher/lower does this user characteristically rate games than everyone else? Raters vary quite a bit in measured "optimism". (Particularly people who only rate one or two games of course.) The standard deviation across all users is about 0.7 of a rating point. The average user optimism is about +0.25 of a rating point, which happens because the more games someone has rated the lower their ratings tend to be so a few pessimistic users can drag all the ratings down. Like this:
Rates # Users Avg Opt Std Dev
1->9 10651 1.03 1.30
->99 11400 0.27 0.67
->999 2876 -0.11 0.56
->9999 10 -0.23 0.51
Then for each ranked game I found the average Optimism for the game's raters. For this bit, I ignored the Optimism rating for any user who rated less than three games, since the granularity of integers would dwarf their optimism measures AND for one-game-raters in particular we'd like to let them have some opinion other than average. Instead I assume anyone with three or less rating has the same Optimism as average everyone else on each of their games.
Then I took the average Optimism and adjusted the game's overall rating, taking into account the Bayesian calculation used on BGG. There's a slight level of approximation because I can't say how many ratings are removed by Aldie's ShillSeeker Filter (tm, pat pend).
The result was a bit of an anticlimax. But you've read this far so you might as well have a look!
-
Joe Grundy
Australia
Sydney
NSW
-
(2000) City Building, Medieval, Tile Placement
Raters: 8907
Avg rater optimism: 0.10 ratings points
Rank change: #43 => #84
Outside shuffles in the top ten games, here's the biggest relative rank demotion, not quite doubling its rank but staying in the top 100. And, incidentally, winding up closer to
#62 => #87 Carcassonne: The City
#91 => #117 Carcassonne: Hunters and Gatherers
#94 => #121 Carcassonne: The Castle
-
-
Joe Grundy
Australia
Sydney
NSW
-
(1995) Civilization, Trading, Modular Board
Raters: 9302
Avg rater optimism: 0.09 ratings points
Rank change: #24 => #46
Since it turns out that on average raters of fewer games are more optimistic, games with many many ratings are more likely to be boosted with a little optimism. So it's no surprise to see the most rated game on BGG here on this list. (Puerto Rico certainly has a little optimism behind it too, but it turns out not enough to drop it from #1 when it's "corrected".)
-
-
Joe Grundy
Australia
Sydney
NSW
-
(2004) Wargame, World War II, Modular Board, Campaign/Battle Card Driven, Co-operative Play, Hand Management, Dice Rolling
Raters: 3753
Avg rater optimism: 0.01 ratings points
Rank change: #19 => #30
-
-
Joe Grundy
Australia
Sydney
NSW
-
(2003) Electronic, Bluffing, Deduction, Paper-and-Pencil
Raters: 145
Avg rater optimism: 0.41 ratings points
Rank change: #1131 => #1757
-
-
Joe Grundy
Australia
Sydney
NSW
-
(2004) Transportation, Trains, Travel, Hand Management, Point to Point Movement
Raters: 5673
Avg rater optimism: 0.03 ratings points
Rank change: #33 => #51
-
-
Joe Grundy
Australia
Sydney
NSW
-
(2005) Card Game, Humor, Fantasy, Role Playing
Raters: 119
Avg rater optimism: 0.35 ratings points
Rank change: #632 => #968
-
-
Joe Grundy
Australia
Sydney
NSW
-
(2003) Wargame, Miniatures, Fantasy, Adventure, Movies/TV/Radio theme, Novel-based, Area-Impulse, Dice Rolling
Raters: 673
Avg rater optimism: 0.20 ratings points
Rank change: #606 => #900
-
-
Joe Grundy
Australia
Sydney
NSW
-
(2004) Card Game, Humor, Fantasy, Science Fiction, Horror, Trading, Role Playing, Dice Rolling, Variable Phase Order
Raters: 327
Avg rater optimism: 0.26 ratings points
Rank change: #875 => #1285
-
-
Joe Grundy
Australia
Sydney
NSW
-
(1995) Fantasy, Adventure, Tile Placement, Co-operative Play, Role Playing
Raters: 419
Avg rater optimism: 0.13 ratings points
Rank change: #229 => #334
-
-
Joe Grundy
Australia
Sydney
NSW
-
(2002) Civilization, Ancient, Set Collection, Trading, Hand Management
Raters: 453
Avg rater optimism: 0.15 ratings points
Rank change: #262 => #379
-
-
Joe Grundy
Australia
Sydney
NSW
-
The next ten biggest relative movers:
Stratego: Star Wars
(2002) Fighting, Fantasy, Science Fiction, Secret Unit Deployment
Raters: 112
Avg rater optimism: 0.42 ratings points
Rank change: #1600 => #2313
Star Munchkin
(2002) Card Game, Humor, Science Fiction, Hand Management
Raters: 564
Avg rater optimism: 0.21 ratings points
Rank change: #851 => #1215
RoboRally
(1994) Racing, Science Fiction, Simultaneous Action Selection
Raters: 3646
Avg rater optimism: 0.00 ratings points
Rank change: #66 => #94
Dungeoneer: Den of the Wererats
(2004) Card Game, Dice, Fantasy, Exploration, Adventure, Action Point Allowance System, Tile Placement, Modular Board, Variable Player Powers, Hand Management, Dice Rolling
Raters: 174
Avg rater optimism: 0.23 ratings points
Rank change: #671 => #948
Risk: Godstorm
(2004) Dice, Fighting, Mythology, Area-Impulse
Raters: 437
Avg rater optimism: 0.18 ratings points
Rank change: #688 => #966
Carcassonne: The City
(2004) City Building, Medieval, Tile Placement
Raters: 1051
Avg rater optimism: -0.01 ratings points
Rank change: #62 => #87
Dragon Strike
(1993) Fighting, Fantasy, Exploration, Adventure, Variable Player Powers, Role Playing
Raters: 147
Avg rater optimism: 0.33 ratings points
Rank change: #1631 => #2282
Dungeons & Dragons: The Fantasy Adventure Board Game
(2002) Fighting, Miniatures, Fantasy, Exploration, Adventure, Horror, Modular Board, Variable Player Powers, Co-operative Play, Role Playing, Dice Rolling
Raters: 260
Avg rater optimism: 0.20 ratings points
Rank change: #698 => #975
Heroscape Master Set: Rise of the Valkyrie
(2004) Miniatures, Fantasy, Science Fiction, Adventure, Modular Board, Hex-and-Counter, Dice Rolling
Raters: 2204
Avg rater optimism: 0.08 ratings points
Rank change: #118 => #164
Risk: Star Wars; The Clone Wars Edition
(2005) Wargame, Movies/TV/Radio theme, Dice Rolling
Raters: 327
Avg rater optimism: 0.21 ratings points
Rank change: #815 => #1131
So when it comes down to it, the ratings are not being massively shaken up by any coincidences of optimism.
-
-
Joe Grundy
Australia
Sydney
NSW
-
THE TOP TEN...
... are still all in the top ten! But, technically, along with the biggest rank drop the two biggest relative rank climbers of all games, Twilight Struggle and Die Macher, are here because the absolute rank numbers are so low.
Puerto Rico
(2002) Economic, City Building, Variable Phase Order
Raters: 8055
Avg rater optimism: 0.02 ratings points
Rank change: #1 => #1
Tigris & Euphrates
(1997) Civilization, Tile Placement
Raters: 4905
Avg rater optimism: -0.06 ratings points
Rank change: #2 => #2
Power Grid
(2004) Economic, Auction/Bidding
Raters: 4187
Avg rater optimism: -0.08 ratings points
Rank change: #3 => #3
Caylus
(2005) Economic, City Building, Medieval
Raters: 3524
Avg rater optimism: -0.08 ratings points
Rank change: #4 => #4
Twilight Struggle
(2005) Wargame, Political, Campaign/Battle Card Driven
Raters: 1020
Avg rater optimism: -0.20 ratings points
Rank change: #8 => #5
Die Macher
(1986) Economic, Political, Auction/Bidding
Raters: 1098
Avg rater optimism: -0.19 ratings points
Rank change: #9 => #6
El Grande
(1995) Political, Auction/Bidding, Area Movement
Raters: 3770
Avg rater optimism: -0.07 ratings points
Rank change: #6 => #7
Age of Steam
(2002) Transportation, Trains, Tile Placement, Pick-up and Deliver, Commodity Speculation
Raters: 1928
Avg rater optimism: -0.18 ratings points
Rank change: #10 => #8
The Princes of Florence
(2000) City Building, Renaissance, Auction/Bidding
Raters: 3439
Avg rater optimism: -0.09 ratings points
Rank change: #7 => #9
BattleLore
(2006)
Raters: 1387
Avg rater optimism: -0.01 ratings points
Rank change: #5 => #10
-
-
Joe Grundy
Australia
Sydney
NSW
-
BUT WHAT ABOUT THE OTHER END OF THE SPECTRUM?
Are there any games being dragged down by Marvin-esque raters seeing the bad side of everything?
Outside the top ten, here is the biggest relative improver after adjusting for pessimism.
(2004) Wargame, Hand Management
Raters: 481
Avg rater optimism: -0.26 ratings points
Rank change: #67 => #45
-
-
Joe Grundy
Australia
Sydney
NSW
-
(2004) Wargame, Civilization, Political
Raters: 945
Avg rater optimism: -0.22 ratings points
Rank change: #34 => #23
-
-
Joe Grundy
Australia
Sydney
NSW
-
(2003) Negotiation, Farming, Tile Placement, Auction/Bidding
Raters: 1146
Avg rater optimism: -0.19 ratings points
Rank change: #77 => #53
-
-
Joe Grundy
Australia
Sydney
NSW
-
(2005) Wargame, Napoleonic, Action Point Allowance System, Secret Unit Deployment, Area Movement
Raters: 407
Avg rater optimism: -0.24 ratings points
Rank change: #59 => #42
-
-
Joe Grundy
Australia
Sydney
NSW
-
(2004) Fantasy, Auction/Bidding, Variable Player Powers
Raters: 586
Avg rater optimism: -0.25 ratings points
Rank change: #120 => #86
-
-
Joe Grundy
Australia
Sydney
NSW
-
(2005) Transportation, Economic, Industry / Manufacturing
Raters: 290
Avg rater optimism: -0.28 ratings points
Rank change: #114 => #82
-
-
19.
Board Game: Tichu
[Average Rating:7.73 Overall Rank:36]
Joe Grundy
Australia
Sydney
NSW
-
(1991) Card Game, Trick-taking, Partnerships
Raters: 1168
Avg rater optimism: -0.18 ratings points
Rank change: #36 => #26
-
-
Joe Grundy
Australia
Sydney
NSW
-
(1999) Transportation, Civilization, Economic, Tile Placement, Pick-up and Deliver, Modular Board
Raters: 467
Avg rater optimism: -0.22 ratings points
Rank change: #40 => #29
-
-
Joe Grundy
Australia
Sydney
NSW
-
(2002) Wargame, Medieval, Secret Unit Deployment, Simultaneous Action Selection, Area-Impulse, Area Movement
Raters: 999
Avg rater optimism: -0.18 ratings points
Rank change: #26 => #19
-
-
22.
Board Game: Medici
[Average Rating:7.18 Overall Rank:223]
Joe Grundy
Australia
Sydney
NSW
-
(1995) Renaissance, Set Collection, Auction/Bidding
Raters: 1444
Avg rater optimism: -0.18 ratings points
Rank change: #97 => #71
-
-
Joe Grundy
Australia
Sydney
NSW
-
The next ten biggest relative movers:
Crokinole
(2000) Action/Dexterity, Partnerships
Raters: 784
Avg rater optimism: -0.18 ratings points
Rank change: #38 => #28
Princes of the Renaissance
(2003) Economic, Negotiation, Medieval, Renaissance, Auction/Bidding, Variable Player Powers, Dice Rolling
Raters: 753
Avg rater optimism: -0.19 ratings points
Rank change: #76 => #56
Bridge
(1925) Card Game, Trick-taking, Auction/Bidding, Partnerships
Raters: 604
Avg rater optimism: -0.21 ratings points
Rank change: #103 => #76
Logistico
(2003) Transportation, Abstract Strategy, Economic, Trains, Nautical, Industry / Manufacturing, Action Point Allowance System, Pick-up and Deliver, Area Movement, Simulation
Raters: 242
Avg rater optimism: -0.23 ratings points
Rank change: #916 => #683
Imperial
(2006) Economic, Fighting, Area Movement
Raters: 473
Avg rater optimism: -0.20 ratings points
Rank change: #52 => #39
Volldampf
(2001) Trains, Pick-up and Deliver
Raters: 305
Avg rater optimism: -0.26 ratings points
Rank change: #287 => #216
Street Illegal
(2002) Card Game, Racing, Sports, Bluffing, Modular Board, Simultaneous Action Selection, Hand Management
Raters: 145
Avg rater optimism: -0.27 ratings points
Rank change: #1947 => #1469
Keythedral
(2002) City Building, Medieval, Auction/Bidding
Raters: 1023
Avg rater optimism: -0.17 ratings points
Rank change: #90 => #68
Fresh Fish
(1997) Transportation, Tile Placement
Raters: 475
Avg rater optimism: -0.26 ratings points
Rank change: #421 => #320
Europa 1945-2030
(1998) Negotiation, Political, Voting
Raters: 201
Avg rater optimism: -0.23 ratings points
Rank change: #1798 => #1373
-
-
|
|
Posted Sat Mar 3, 2007 7:47 am
Sydney
NSW
Randy, you're Optimism value is -1.01.
People who have rated roughly as many games as you average about -0.35, standard deviation 0.48 ie you seem to be notably hard on the games you rate. But then, I can guess that from your Game Rating Distribution chart:
http://www.boardgamegeek.com/user/Randy+Cox
(Although all those "4"s might have been Monopoly variants or something.)
For myself, my Optimism value is 0.14
People at my lower levels of ratings offerings average about 0.03, standard deviation 0.57 so it seems I'm a little fluffier than average, but within the "most people" box.
Clemson
South Carolina
I've thought of a convoluted way to try this, given that simply plucking the last login date seems to be impossible...
What if you took a list of the 10 most popular games of each of the past five or six years (most popular = most ratings)? Then, give each person a score based on how many games they have rated in this group.
Say one person has rated 10 games from 1999, 7 from 2000, 3 from 2001, 2 from 2002, and 1 from 2005. That would indicate that the person sort of dropped out of the gaming world somewhere in the past half-decade (precisely where would be left to the arcane forumla one devises). That person would be given only partial (or no) credit when trying to figure out the favorite games of those who frequent BGG and those who still purchase, play, or rate games.
Sydney
NSW
(I think I'd "fail" your test.
Shouldn't we allow that if they've rated anything from 2006 then they're still in business? Even then I wasn't sure I've rated any game new from 2006... turns out I've rated three games from 2006 and two from 2005.
Your idea has merit if I want to filter for recent players only. I'll think on what I can do with it.
How about "Puerto Rico (2002) opinion by year of last rated game"?
LatestRatedGameYear, NumberOfRaters, AverageRating
2002 538 8.79
2003 309 8.76
2004 1074 8.67
2005 1797 8.53
2006 4290 8.39
Compared to, say, Tigris & Euphrates (1997)
2001 52 8.27
2002 129 8.57
2003 148 8.49
2004 511 8.30
2005 987 8.30
2006 2961 8.23
2007 39 7.52
Compared to, say, Caylus (2005)
2005 764 8.62
2006 2724 8.20
Or especially compare El Grande (1995)
2001 48 8.39
2002 98 7.95
2003 110 8.03
2004 366 8.09
2005 667 8.19
2006 2390 8.12
with The Settlers of Catan (1995)
2000 136 8.57
2001 124 8.20
2002 401 8.22
2003 397 8.16
2004 1301 8.10
2005 2045 7.85
2006 4618 7.57
Note: I left out the handful of people who have already rated a 2007 game because in every case the 2007 rater average is much less than the "usual" averages for each game. I also left out an early year here and there where the number of raters is too small to get a reliable answer.
Well it certainly shows ratings for a game decline gently over time. There's a list in this!
Clemson
South Carolina
LatestRatedGameYear, NumberOfRaters, AverageRating
2002 538 8.79
2003 309 8.76
2004 1074 8.67
2005 1797 8.53
2006 4290 8.39
with The Settlers of Catan (1995)
2000 136 8.57
2001 124 8.20
2002 401 8.22
2003 397 8.16
2004 1301 8.10
2005 2045 7.85
2006 4618 7.57
I like this approach. I think that Puerto Rico is closer to a "true" 8.39 than to its actual 8.49 and that Settlers is closer to a "true" 7.57 than to its actual 7.80. There is some merit in looking at ratings of people who have rated at least some recent games.
Of course, this info you show indicates that a huge, huge majority of people must be out there gobbling up the latest game to float by, playing it, and rating it. I would have thought that surely more than half the people who had rated a particular game would have rated no games in 2006. Boy was I wrong!
It would be interesting to see how these figures stack up for Acquire or Up Front or even Drunter & Druber. You know, older stuff that isn't in the "must play" list for noobs.
Sunderland
Massachusetts
Good question. But then, how would you get all of the ratings of every user without opening every user page, or every game page? Via the XML API, presumably, but still requiring at least one hit per game or per user. Actually, I don't see any way in the XML API to get all ratings for a game, though all comments is easy. Is there an undocumented option to do so?
I suppose you can fetch them using this screen: http://www.boardgamegeek.com/gamestats.php3
It looks like the format for the POST is not complicated.