Home > Gaming > Metacritic to Start Rating Developers

Metacritic to Start Rating Developers

March 28th, 2011

Metacritic Home Page ScreenshotOver the weekend Industry Gamers reported that meta scoring site, Metacritic will begin to rate individual game developers. According to the article the ranking will be based on aggregate scores from the individual titles from each developer. The rankings will show the highest, lowest, and average score of all the games for each developer.

While on this surface this may seem like a quantifiable metric we need to ask ourselves what this score is really measuring. In other words by averaging the scores of the games each developer has worked on can we get a reliable metric that can be used to predict a future outcome?

Games are a collaborative effort and many factors make up the decisions that go into the production of a completed game, such as budgets and deadlines, just to name a few. All of these additional external factors add variability to the data. This variability adds to the likelihood that the relationship between developer and game score is not sufficiently explained by the developer alone.

Depending on the amount these external factors come into play ratings for some developers may be better than that of others, but all of this will depend on the algorithm Metacritic will use to measure this relationship. In the end, my greatest fear is that a rating system like this will detour innovation for fear of a poor score, especially for developers new to the industry. Games, like films, books, etc. should be scored on their individual merits. Change any part of a team or the dynamics the team worth within and the rules the relationship was based on become void. We will just have to wait and see how the industry and gaming community take to this new score, how it will evolves over time and ultimately how reliable it is.

***UPDATE 03/29/2011***

According to gamesindustry.biz Metacritic has pulled the controversial individual developer ratings from its site indicating in the company’s official blog that “[the score is a] work in progress and is not nearly as comprehensive as it needs to be to accurately provide a career score for these individuals.”

***UPDATE 03/31/2011***

Well I looks like this issue is closed for the moment. Today gamesindustry.biz is reporting that “Metacritic has stated that it will not bring back scores for individual developers after a week in which it was roundly criticised by the games industry for rating individuals using incomplete or out-of-date information.”

Kudos to Metacritic for admitting the flaws in their rating system and resolving this issue so quickly.

Categories: Gaming Tags:
  1. June 5th, 2011 at 13:47 | #1

    F*ckin’ amazing things here.I am very glad to see your article.Thanks a lot and i am looking forward to contact you.Will you please drop me a mail?

Comments are closed.