22 May 2009

Analysing the Analysists


For a while now I have been debating the value and validity of critic review scores. It seems as if the subject has come under more & more scrutiny from the community in direct proportion to the popularity of sites such as Metacritic. This is a fairly banal subject now, however review scores are still one of the main factors which gamers base their purchases on.
In my opinion, review scores are not meant to be the foundations...
...they are there as merely a guide to quality, judged by writers with genuine experience in the field. Personally, if a game is being released which I believe I'll like (due to the universe, genre, gameplay, etc), I'll purchase it with little regard to review scores. Cleaning the egg from my face if I don't like it will be my duty alone, without a scape goat to blame. However people who base their entire opinions on review scores are neutering their own opinions and beliefs.
But it can also go the other way; when people disregard review scores entirely they are ignoring a pool of knowledge who in most cases are unbiased and really do know what they're talking about. There is a modest middle ground where you can be opinionated, but are also a good listener.

This brings me nicely onto my next point, up until now I've been talking about consumer reaction to review scores, but this twisted tale of subjection ventures much higher in the journalistic hierarchy. I'm going to assume we all heard about Jeff Gerstmann controversy concerning his departure from Gamespot. It was rumored that in this case external pressure from a publisher caused Jeff to lose his job due to a review he had submitted for a particular game. I believe it was slightly blown out of proportion (like most news), however this is still a prime example of the current importance of reviews and the scores which summarise them.

However, aren't we all applying a little too much emphasis on the score at the end? Many times I've skipped the few pages of text, be this due to time constraints or my own laziness, either way I can't argue against how handy the number is. But that's it, it's just a time saving device, you don't need a degree in Mathematics to realise that a game which scores 40% is not as good as one which scores 80%. But it's why these games are what they are, and how they achieve it, which is explained in the text of the review. Sights such as Metacritic combine a number of these time saving devices into an even time savier device, which is relatively needless.

I'm finding it difficult to delve much deeper into the subject the more I try and apply logic. I find it hard comparing two reviews of the same game when one uses a four star ("but technically nine") system while the other employs a variety of letters.

The scale Metacritic uses also intruiges me, as it combines the aforementioned array of different scoring methods into one, 0-100 range. What I don't understand is what constitutes the difference between a 75 rated game and a 76 rated game. What definable feature of the latter gives it that extra single point over the former? I believe this is the crux of the matter with review scores and their value. Who can define a definate value for a game score, when the whole point of reviews is that they are opinion based, what sort of system is that?

3 comments:

  1. Personally... I go through a process of elimintaion before i buy a game. 1st I ask myself what genre of game do I want (I have preferences but overall I'm pretty varied). Deciding that I then ask myself whether or not I want a long term purchase (borderline commitment) or a short term purchase(to past the time). 3rd thing is budget.

    I usually have my ear to the ground in terms of games so i tend to know whats coming out and what would interest me. I'll always prefer a sequel or a new release from a prefered developer so they're usually on my list first. But a few striking games may make it on to said list.By the time i go to the store my mind is usually set on what game i want.

    To me reviews are a resource, not a true measure of a games interest to me. Granted i'll avoid anything below 5/10 or 2 stars or whatever, but i'd say thats a general consensus. 10's 9's 8's or whatever, to me they have equal potential to entertain (and a slight chance not to interest me at all). Altough thats my standing i know a couple of people who treat reviews as a scientific measure, and wont touch anything without a score of above average. Then the exteme are people who buy a game cause they "like the name".

    Also... Don't underestimate the power of statistics on the simple minded. Statistics are bull when it comes to such fine differences but it makes them seem more "legit" to the average person. Statistics are the new craze, everybody just eats em up.

    ReplyDelete
  2. But why do you avoid anything below 5/10? And what constitutes "average"? Realistically it should be said 5 out of 10, however to you a game has to be 8 or higher to have an "equal potential to entertain". Does that make a 7 game "average"?

    ReplyDelete
  3. 10-5 is the range i tend to buy games, and its not from one source i take these values. Some sources are more reputable then others and their scoring and reviews carry more weight. But ultimately i follow my gut, which over the years has been refined so i can eye a good game or a game with good potential before it's launched, i look past the bullshit the sales men regurgitate on to us and look for it's potential within the games visual, aural or story parameters. I watch video demos and read previews and then decide wheter or not it may interest me.

    I personally hate buying average quality games... those are games that i buy play and maybe finish only to sell it off or have gather dust for all eternity. However giving that average is relative to ones own disposition towards games means i cant give it a value, ive played games with a score of 8 or 9 (which is consistant with most reviews) and found it to be average. Ive bought games that some considered to be average only to have hours of entertainment from it and possibly still play.

    As i said already i treat reviews as a tool and not as a matter of truth, or scientific fact. I might not share the same opinions on a game that he/she (more likely he) does. Therefore i can only take their opinions as one perspective of many. While they are told to be unbiased it's very rare one would find an unbiased review. So my final call on what game i buy is a gut instinct, which as i said has been fairly well refined over the years. It rarely steers me wrong. Can't say the same about peer influences, many a games i bought that i regret due to "recommondations".

    Oh and "realistically" an average would be the some of all the reviews scores divided by the number of reviews. NOT 5/10

    ReplyDelete