9.5 is a great way to go, I hate to say 10, ethics you know

With the recent little Eidos kerfuffle making the headlines here, I feel like addressing the issue.

Is the games reviewing industry corrupt? No.

It suffers from the same problems endemic to criticisms of other media forms. Roger Ebert has spoken of going to events, and having publications sign up for quotes to run praising a certain movie. Movies that are expected to show poorly with the critics simply aren't screened early. What both industries here are the victim of is a simple and logical desire of the producers to have people like their product. Loosely speaking, this is their job- making sure the game gets recieved nicely. A high metacritic score is a key component of this, and making sure early reviews are positive is a means to that end. There is absolutely nothing wrong with this, and those of you that jump to vilify these people should take a moment to consider your actions.

Apparently, Ubisoft dun liek you giving their premiere games 4's. Pricks.
They either spent the last couple years of their lives working on this product, or are closely working with people who did. Unless you're working on a POS licensed game like Bolt, you're going to be understandably proud of that work. A high score is validation of that time and effort. It's the human condition to strive to excel, even if it means trying to game the tables in your favor a bit.

Of course, usually responsible journalism is there to slap the balance of power right back to where it should be- problems arise when perhaps the journalistic community willingly lets a publisher game the system a bit. Perhaps they think,"Hey is giving Assassin's Creed a low score really worth having Ubisoft completely cut us off from all their events? Honestly, the audience probably really won't even appreciate our earnesty, so why is it worth fighting Ubi over?" This puts us in a bit of a sticky siuation here, as both sides are dependent on the other.
  1. The games press earns their money in part by advertising for games they cover
  2. The games publisher and developer earn their money based in part off of reviews, and get bonuses based off of the Metacritic average
So what you see there is something that could EASILY become a "You rub my back, I'll rub yours" situation, wherein publications run good reviews in exchange for advertising dollar. Mutually beneficial. No losers, except for the consumers. But that's not how it works

Interesting
Luckily for us, this tenuous balance of power is usually maintained. The only time we ever hear about it is when an outlet lets it slide too far, and you get a Gerstman situation. Just because that's all we hear about doesn't mean it's the status quo. Every time we hear about it is because something exceptional has happened. Something that breaks the normalcy of the situation. The relationship between publishers and jourmalists is obviously not ideal, as it runs on the much derided "honor system". I won't try to judge this relationship absolutely- I won't even presume to be entirely educated in the subject, after all, I'm not a games journalist. Just a guy who finds the business dynamics fascinating.

Ultimately, it would be easiest for journalists just to hand out high scores like candy to appease publishers, and stay secure in their jobs. And this does happen of course- when you see Play magazine handing out 9.0's to Lair, something's fishy. However, journalists (or "professional enthusiasts" as the 1up folks call it) usually have the integrity enough to avoid this pitfall. We know this simply by observing the review scores. EGM gave Assassin's Creed a 4, and paid the consequences. Jeff gave Zelda an 8.8, despite it not technically being in the "AAA" tier. You've seen big games like Mirror's Edge, which no doubt have had tons of marketing dollar thrown behind them recieve 7's. Observe the symptoms, and you can diagnose the disease. Or lack of, in this case. A few isolated cases do not condemn the whole industry, a fact well worth remembering.

The title comes from the musical crazy of Shawn Elliot- that particular piece kicked off the 12/5/07 GFW Radio, which has some interesting insights into this very topic.
17 Comments
18 Comments
Posted by Lies

With the recent little Eidos kerfuffle making the headlines here, I feel like addressing the issue.

Is the games reviewing industry corrupt? No.

It suffers from the same problems endemic to criticisms of other media forms. Roger Ebert has spoken of going to events, and having publications sign up for quotes to run praising a certain movie. Movies that are expected to show poorly with the critics simply aren't screened early. What both industries here are the victim of is a simple and logical desire of the producers to have people like their product. Loosely speaking, this is their job- making sure the game gets recieved nicely. A high metacritic score is a key component of this, and making sure early reviews are positive is a means to that end. There is absolutely nothing wrong with this, and those of you that jump to vilify these people should take a moment to consider your actions.

Apparently, Ubisoft dun liek you giving their premiere games 4's. Pricks.
They either spent the last couple years of their lives working on this product, or are closely working with people who did. Unless you're working on a POS licensed game like Bolt, you're going to be understandably proud of that work. A high score is validation of that time and effort. It's the human condition to strive to excel, even if it means trying to game the tables in your favor a bit.

Of course, usually responsible journalism is there to slap the balance of power right back to where it should be- problems arise when perhaps the journalistic community willingly lets a publisher game the system a bit. Perhaps they think,"Hey is giving Assassin's Creed a low score really worth having Ubisoft completely cut us off from all their events? Honestly, the audience probably really won't even appreciate our earnesty, so why is it worth fighting Ubi over?" This puts us in a bit of a sticky siuation here, as both sides are dependent on the other.
  1. The games press earns their money in part by advertising for games they cover
  2. The games publisher and developer earn their money based in part off of reviews, and get bonuses based off of the Metacritic average
So what you see there is something that could EASILY become a "You rub my back, I'll rub yours" situation, wherein publications run good reviews in exchange for advertising dollar. Mutually beneficial. No losers, except for the consumers. But that's not how it works

Interesting
Luckily for us, this tenuous balance of power is usually maintained. The only time we ever hear about it is when an outlet lets it slide too far, and you get a Gerstman situation. Just because that's all we hear about doesn't mean it's the status quo. Every time we hear about it is because something exceptional has happened. Something that breaks the normalcy of the situation. The relationship between publishers and jourmalists is obviously not ideal, as it runs on the much derided "honor system". I won't try to judge this relationship absolutely- I won't even presume to be entirely educated in the subject, after all, I'm not a games journalist. Just a guy who finds the business dynamics fascinating.

Ultimately, it would be easiest for journalists just to hand out high scores like candy to appease publishers, and stay secure in their jobs. And this does happen of course- when you see Play magazine handing out 9.0's to Lair, something's fishy. However, journalists (or "professional enthusiasts" as the 1up folks call it) usually have the integrity enough to avoid this pitfall. We know this simply by observing the review scores. EGM gave Assassin's Creed a 4, and paid the consequences. Jeff gave Zelda an 8.8, despite it not technically being in the "AAA" tier. You've seen big games like Mirror's Edge, which no doubt have had tons of marketing dollar thrown behind them recieve 7's. Observe the symptoms, and you can diagnose the disease. Or lack of, in this case. A few isolated cases do not condemn the whole industry, a fact well worth remembering.

The title comes from the musical crazy of Shawn Elliot- that particular piece kicked off the 12/5/07 GFW Radio, which has some interesting insights into this very topic.
Posted by brukaoru
The games publisher and developer earn their money based in part off of reviews, and get bonuses based off of the Metacritic average
This is an awful practice. This really needs to stop. Metacritic is ridiculous when they turn a 5 star rating system into a rating based off of a 1-100 rating, such as a 3 star rating turning into a 60. Developers should not have their money based on such a rating scale.

Good point about not all reviews being paid off to elevate scores. A few maybe, but you can trust a majority of reviews won't do such a thing.
Edited by SmugDarkLoser
brukaoru said:
"
The games publisher and developer earn their money based in part off of reviews, and get bonuses based off of the Metacritic average
This is an awful practice. This really needs to stop. Metacritic is ridiculous when they turn a 5 star rating system into a rating based off of a 1-100 rating, such as a 3 star rating turning into a 60. Developers should not have their money based on such a rating scale.

Good point about not all reviews being paid off to elevate scores. A few maybe, but you can trust a majority of reviews won't do such a thing."
True.  And isn't a 3 star more well translated to a 70% ish game?'
Either way, they kind of do out of 5 stars/ letter scale for a reason.  Metacritic is a joke anyhow.  Niche games like JRPGs get screwed in the process and well, really anything that takes a risk (looking at banjo)


- Anyway, you don't completely, but don't make the game developers seem like the bad guys in this.  You're sort of making it one sided.  You have to understand, some of these guys have worked on their game for a couple of years and truly think they've made something spectacular (and really, most games are in some respect) only to see some low basically bashing scores
Posted by Bulldog19892

Damn good blog post. Well said, well written.

Posted by DualReaver

I liked the picture of Jeff. :D

Edited by daniel_beck_90

well people need money so they can pay their bills right ?  unfortunately this may sometimes lead to  what you previously mentioned , but even then we can distinguish a good game from a bad game ourselves  by observing different reviews and figuring the main content of the game ourselves . Halo 3 received   positive reviews but Did  I buy that  game ?  No  , becasue after reading all those extremely positive reviews I understand that this game is shallow eventhough all the reviews were praising the game  .

Posted by Sharkboy242

HALO?

Posted by Lies
SmugDarkLoser said:
- Anyway, you don't completely, but don't make the game developers seem like the bad guys in this.  You're sort of making it one sided.  You have to understand, some of these guys have worked on their game for a couple of years and truly think they've made something spectacular (and really, most games are in some respect) only to see some low basically bashing scores
"
I tried to avoid that, certainly neither side is blameless, and neither side is totally responsible.
I though I managed to dispel any ideas that developers were to blame here:
They either spent the last couple years of their lives working on this product, or are closely working with people who did. Unless you're working on a POS licensed game like Bolt, you're going to be understandably proud of that work. A high score is validation of that time and effort. It's the human condition to strive to excel
Posted by clarke0

I have no doubt that the game-reviewing industry is corrupt. I think the gaming community is part of the problem, as they will group up to hate on a game that got a poor review score from their favorite site. The main reason why I like this site is because I know the reviewers are being honest about the experience they had with the game. Unfortunately people still bitch about the amount of stars games get on the forums.

Posted by LiquidPrince

Assassin's Creed was a good game.

Posted by DualReaver
LiquidPrince said:
"Assassin's Creed was a good game."

Please, excuse me, but I have to ask: Are you only being nice because you want something?
Posted by Psynapse

Great write up bud.

I didn't mind Assassins Creed either, wasn't my favorite game, but no way did it deserve a 4 :)

Posted by Unreal_dro
LiquidPrince said:
"Assassin's Creed was a good game."
no
Posted by Champy
Unreal_dro said:
"LiquidPrince said:
"Assassin's Creed was a good game."
no"
yes
Posted by Lies
Champy said:
"Unreal_dro said:
"LiquidPrince said:
"Assassin's Creed was a good game."
no"
yes"
The point. You are missing it.
Posted by Bellum

I blame the developers for a lot of things, but I don't blame them for the state of the review business. I blame the reviewers. No, no amount of underhanded dealing is acceptable. I also perhaps put the majority of blame on the audience. No, your new favorite game does not deserve a 10.

Posted by Lies

It's important to remember that devs are often not directly related with these fiasco's, it's the publishers and PR firms that tend to get themselves into trouble with the games press, and blame you shift towards the developer is almost always mis-targeted.

Posted by Sparky_Buzzsaw
Moderator Online