Now that we’re well into the video game industry’s silly season with tons of hotly anticipated games coming out, there’s inevitably a lot of discussion that comes up about how they are received critically. I read an interesting article the other day, in which COO of Double Fine Justin Bailey (yes, that is his real name) made some pointed comments about how he perceives the impact of aggregator sites like Metacritic. According to him, how a game fares there has little to no effect on overall sales and is “all but obsolete” in this day and age.
I tend to agree with Bailey to a certain point in that sales of most major releases are not much affected by reviews, unless of course they are exceptionally high or low. Where I take exception however is with his assertion that Metacritic is obsolete. On the contrary, I would argue it’s a resource that, when used properly, provides an invaluable service for people interested in video games. None of the issues with Metacritic are impossible to fix, and here are some things we can all do to maximize its usefulness.
Metacritic: Do Away With ‘Weighted Averages’
If Metacritic is going to serve as a true aggregate of video game criticism, then they need to do away with their ‘weighted average‘. in which they basically calculate the mean score but allocate weight “based on their quality and overall stature.” Essentially, this gives a louder voice to large sites with big audiences.
No doubt, this system is ostensibly in place to minimize average-skewing scores that can drag a game down (I’ll get to you in a minute, User Reviews). There are two problems with that approach however; first, it assumes that reviewers from larger sites won’t stray from the critical mass, and second, it suppresses alternative viewpoints on a particular titles, which is an essential feature of a healthy online community.
Metacritic maintains control over which outlets they use to calculate their aggregates, so ostensibly they are engaging in some quality control already by creating a semi-exclusive club whose point of view is valued over everyone else. Further segregating that group only widens the gulf and skews results. Instead, let the data stand on its own.