Log in or sign up to comment
231 Comments
Posted by LiquidSwords

Bloggers, Metacritic, and Rotten Tomatoes, they are all the same: Worthless.

Posted by Boiglenoight

I use Metacritic for professional reviews. If I'm interested in a game and it's gotten a low 80, I may look to user reviews for perspective but review bombs are obvious. If light of one, I'll just rely on impressions from game forums.

Posted by confideration

So basically Metacritic says: it's the Internet. SORRY.

Posted by Owlright

I don't use Metacritic except very rarely to check professional reviews, but I know a lot of my friends do put stock in the scores on Metacritic. It sucks for smaller studios like this that such a thing is going to affect their sales and perception. I don't think asking for reviews is a good way to deal with the problem, but realistically what else can they do? Sucky situation :|

I'm not a fan of Metacritic's set-up, and I wish that they'd employ a system similar Rotten Tomatoes, which seems to work fairly well. Not perfectly, mind, but certainly better than Metacritic's current set-up.

Posted by Undeadpool

@jayjonesjunior: It deserved to have people spam 0/10 reviews before the game even came out? How the game turned out is irrelevant at that point, cause that's just straight-up lying. Also didn't make one lick of difference for the game itself.

Posted by Kartana

user-scores --> Lol!

Posted by NcJoker

usually when you can't find who is responsible, it is the party with something to gain. in this case, that would be the companies paying the developers. if they are able to anonymously lower the meta critic rating of the game, then they are able to save money through paying lower bonuses, all while taking no risk. So, why wouldn't companies engage in this practice? The solution is for developers to refuse contracts which rely on a meta critic rating, rather than waiting for meta critic to solve the issue. Or, require account verification

Posted by imsh_pl

I'm pretty sure that not many people take the users reviews seriously, there are like only 0s and 10s out there.

Posted by spartan1017

metacritic is stupid. gamerankings is bettter

Posted by CharlesSurge

No gamer I know cares about a Metacritic score.

Posted by roughplague

it's a silly thing of course, but it DOES factor into sales, they can lose a lot by a super low (spammed) user critic rate, because some people see a score, and they go "oh, I guess that's a shit game, not buying that" regardless of if it's critics or spam assholes.

Posted by gbrading

Look no further than a little game called Kane and Lynch if you want to see a prime example of this sort of score skewing in action.

Posted by NegativeCero

I understand that he wasn't trying to bribe positive reviews, but that is how I took it when I first read it. The problem I have is how is someone supposed to write a review for a game they haven't played, at least I assume they haven't since they would be going for that code. If they are doing it for the code, clearly the "promotion" worked. I get that user reviews are dumb anyway, but I hate the way they went about this.

Posted by DonPixel

Metacritic is troll country

Edited by benjaebe

@ProfessorEss said:

As long as (pretty much) every reviewer out there is playing right into Metacritic's hands by sending them numbers that they know full well are going to be mathematically converted to a 100 point scale I can't see how we can place all the blame squarely on Metacritic.

You can't blame Sessler or Giant Bomb for "sending" Metacritic numbers which are reinterpreted without their consent, nor should you expect them to change their methods of reviewing because Metacritic exists and is perverting their work.

EDIT: To make the point more clear, they aren't agreeing to having their reviews appear on Metacritic. Metacritic aggregates the reviews and converts them without any reviewer input.

Posted by BisonHero

Unless user reviews are the ONLY form of reviews available to the website (like with Amazon), I don't think any website really benefits from user-submitted scores. Too much potential for trolling, or for idiots with an axe to grind who just rate something as either a 0/10 or a 10/10, despite very few games deserving either of those scores.

Wanna submit a review of a game? Become a games journalist. That takes enough effort that it tends to weed out most trolls.

Posted by jayjonesjunior

@Undeadpool said:

I remember when this exact thing happened for Dragon Age II...and I'm sure it will absolutely happen for Gears of War 3. It's really said that publishers prize User Reviews this highly for this exact reason. Yeah, it wouldn't be terribly smart to just tell your user base you flat-out don't care unless they're "critics," but tying THIS much to them is just primed for this kind of random mayhem.

I'm pretty sure Dragon Age 2 deserved that.

Posted by TadThuggish

The way to fix Metacritic is to never use Metacritic.

Edited by hermes

@WilliamHenry said:

@Vegsen said:

Proves why Metacritic is a BAD IDEA.

Metacritic is not a bad idea, its actually a great idea, hows its used is the problem. How can having a large number of different reviews all in one place be a bad idea? The fact that bonuses are paid based on Metacritic scores and the bad policing of spam reviews are the problems, not Metacritic itself.

All reviews should require text. Just picking a number isn't sufficient enough. All reviewers should also have their reviews approved for a period of time to make sure they're not spam or fanboys.

Metacritic as an aggregation of sites reviews is a great idea. Metacritic as an average of points is a terrible idea.

Other sites like rotten tomatoes seems to get it right by paying attention to the review text more than applying a formula to incompatible sites numeric reviews.

Posted by Six

what score would metacritic get if its scoring system was used on itself and users rated metacritic based on the service it provides?

Posted by GeekDown

I use Metacritic a lot, but I never read the user reviews. I'm not even a registered member and I think that's the case with most of it's userbase. People are going there to see how well a game, album or movie is being received by professional journalists. I don't think developers should worry too much though.

Posted by lacke

@Vinny_Says: I think it's more sad that we can't take user reviews more seriously because of blatant bias.

Posted by Shuborno

@metalsnakezero said:

This is a really broken system if things like this is happening.

It's always been a broken system.

"Statistics" generated without a public methodology are not useful for public discourse.

Posted by Ghost_Cat

We have come to a point in modern society where numbers (and associated color schemes) mean more than in depth discussion. Numbers suddenly suddenly mean everything, and consumers assume them as fact without doing a little bit of detailed researched. Whatever happened to reading?

Edited by ProfessorEss

  @WilliamHenry said: 

Really, you don't understand why scores are needed? Its because people are fucking lazy.

Yeah, and I'm saying reviewers should either say: 
 
A. I believe scores are valuable because a lot of people don't have time or are too lazy to read my review 
or 
B. I don't believe in scores so fuck anyone who's too lazy to read what I think about it. 
Just not: 
C. I don't believe scores, but here's a number just in case you're too lazy to read my review.
  

@Pop  said: 

That's some bullshit, there should just be a no text user score, so that this kind of spam couldn't happen. Cause trolls don't like to spend time writing and stuff. And it sucks that game developers have to depend on metacritic which doesn't have the best system. New whiskey site idea super score game and movie site :P. Put the top men on it right away.

I'd go so far as to say not just text but a minimum character count so at the very least a troll has to hold down his M key for a good 15-20 seconds. And get them off the game's main page. 
Posted by swamplord666

I don't know if it's still the case, but there was a time when exlusive AAA titles would bomb simply for the fact that fanboys from the opposing console would write it just to be trolls.

User reveiws are an awesome concept, but it should never be the end all. And sadly it's the case a little too often :/

Posted by Parsnip

I'm always baffled that the metacritic user ratings matter so much to publishers and devs.

Posted by Grognard66

Stupid fanboys. It's not too hard to figure out what happened here. Two exclusive Summer of Arcade games being targeted for negative reviews? I sense a juvenile Playstation fanboy...

Posted by spilledmilkfactory

I always thought of the user reviews on Metacritic as kind of a joke. Most of them are either 10s or 0s. But it's still bullshit that some people step all over these developers' hard work for no discernable reason. Even if that affects one sale, that's money that an indie dev is losing

Posted by Pop

That's some bullshit, there should just be a no text user score, so that this kind of spam couldn't happen. Cause trolls don't like to spend time writing and stuff. And it sucks that game developers have to depend on metacritic which doesn't have the best system. New whiskey site idea super score game and movie site :P. Put the top men on it right away.

Posted by AceBlack19

Reading metacritic user reviews is an exercise in endurance. The negative Bastion user reviews were filled to the brim with misinformation and lies about the games controls and plot.

While a website where everyone can pitch in their own scores is a great idea in theory, its flawed in practice when idiots who never take the time to actually play the game (or at least click the options menu) decide they're allowed to put in their own two cents.

Posted by sugetipula

I think the oficial Metacritic scores are fine( they do represent the majority of profesional reviewers after all),

but the user ones are bullshit. Whoever says not to listen to the profesional scores and just go by user ratings are idiots.

Posted by kpaadet

Metacritc is not crap, publishers deciding bonuses etc off Metacritc score is crap.

Posted by prestonhedges

@BisonHero said:

@Delta_Ass said:

"Excuse me I'd like to give a user review of 1.0."

"Certainly. May I ask why?"

"For spite."

"Spite?"

"That's right. I don't care for the developer."

"I don't think you can review a game for spite."

"What do you mean?"

"Well if there was some problem with the game, if it were unsatisfactory in some way, then we could do it for you, but I'm afraid spite doesn't fit into any of our conditions for a review."

"That's ridiculous, I want to review it. What's the difference what the reason is?"

You can't review a game based purely on spite."

"Well so fine then... then I don't like it and then that's why I'm giving it a 1."

"Well you already said spite so..."

"But I changed my mind..."

"No... You said spite... Too late."

This is pretty much it. Or it's 4chan morons "doing it for the lulz".

"I don't think the game's any good."

"You're just a butthurt spammer! Troll! Everything's amazing and nothing is bad!"

"Except this. I wasted money on this game."

"If it's bad, why not write a ten-thousand paragraph review about it, huh?!"

"Because I've wasted enough time on this game already? And who reads these reviews, anyway? It's not like I'm getting paid for this shit."

"Troll! Spam! Lulz! Butthurt! BBS Door Games!"

Posted by LLJK_Jasta

@RobertOrri said:

@Mordukai said:

Developers know Metacritic is crap. We know Metacritic is crap. Now someone please let the suits in the publishing house know about that.

Yes.

+1

Posted by RichieJohn

@Meatsim: Agreed. A written review should be required. I also they should be moderated to some degree before they're included in the actual user score.

Posted by WilliamHenry

@ProfessorEss said:

Y'know I'd put a lot more blame on Metacritic if I didn't think the whole reviewing process was kinda jacked from top to bottom.

If reviews are written totally subjectively (because apparently even a modicum of objectivity has been deemed and excepted as impossible) and require a number score even though every reviewer out there seems to be against them, then what's the use of any of it to begin with?

And why do they require number scores? That's my big question, and Giant Bomb is a great example. If Giant Bomb truly doesn't believe scores are important (possibly even detrimental), or that their five point scale doesn't apply to standard mathematical conversions then why is there a score at all? If the stars truly represent what they say then why not simply rate the game with a qualifying statement instead of something that can be so clearly translated into a number?


@benjaebe: I do think Sessler nails it here but he raises the same question. He says "we hate scores but, you know, we have to so..." but no one ever seems to explain why they have to. I realize people have bosses, companies have policies, by how can I accept a reviewer who hates scores as having any integrity if they're putting a score that they don't believe in on every piece that they do? I'm not singling out Giant Bomb or Sessler but they're good examples because they have some clout to try and change this if they really believed this.

As long as (pretty much) every reviewer out there is playing right into Metacritic's hands by sending them numbers that they know full well are going to be mathematically converted to a 100 point scale I can't see how we can place all the blame squarely on Metacritic.
...oh and user reviews, man I can't understand how any pub, dev, or customer would be bothered with that bag of shit.

Really, you don't understand why scores are needed? Its because people are fucking lazy. They don't want to read a review, they just want to see a number and make their decision based on that.

Posted by Undeadpool

@Paul_Is_Drunk: The issue then becomes (beyond "two wrongs don't make a right") what did these people actually hope to accomplish? I remember hearing about the "stealth reviews" and seeing the 0/10 user reviews popping up at around the same time, so I'm not sure you can completely say that one was in reaction to another. I actually think DAII turned out fine (better than Origins in some ways, worse in others), it was just weird seeing THAT many people get THAT dedicated to absolutely ruining this game and then have it amount to nothing. DAII outsold Origins (which is merely a statement of fact and not a measure of quality) and yeah, most people have now just kind of shrugged their shoulders and agreed that it wasn't that great.

Posted by Matoya

Who the fuck buys games based on their metacritic USER reviews?

Posted by N7

@JoeyRavn said:

What worries me most is that if I sincerely dislike a game (for example, Bastion) and give it a low score, my rating will be dismissed as "spam". What an awesome tool Metacritic is.

Either you fucking love a game, or you're just a butthurt loser spammer.

Metacritic has never been good for user reviews. It seems like only the worst of the youtube commenters go there and post even more bullshit. Why are publishers unaware of that?

Posted by BisonHero

@Delta_Ass said:

"Excuse me I'd like to give a user review of 1.0."

"Certainly. May I ask why?"

"For spite."

"Spite?"

"That's right. I don't care for the developer."

"I don't think you can review a game for spite."

"What do you mean?"

"Well if there was some problem with the game, if it were unsatisfactory in some way, then we could do it for you, but I'm afraid spite doesn't fit into any of our conditions for a review."

"That's ridiculous, I want to review it. What's the difference what the reason is?"

You can't review a game based purely on spite."

"Well so fine then... then I don't like it and then that's why I'm giving it a 1."

"Well you already said spite so..."

"But I changed my mind..."

"No... You said spite... Too late."

This is pretty much it. Or it's 4chan morons "doing it for the lulz".

Posted by WilliamHenry

@Vegsen said:

Proves why Metacritic is a BAD IDEA.

Metacritic is not a bad idea, its actually a great idea, hows its used is the problem. How can having a large number of different reviews all in one place be a bad idea? The fact that bonuses are paid based on Metacritic scores and the bad policing of spam reviews are the problems, not Metacritic itself.

All reviews should require text. Just picking a number isn't sufficient enough. All reviewers should also have their reviews approved for a period of time to make sure they're not spam or fanboys.

Posted by Fobwashed

This exact thing is a pretty huge problem in the XBLI section on XBL as well. It seems like any game that reaches the top 60 gets a surge of 1 star ratings almost without fail. Whether this is the work of a small group of individuals trying to bring down the ratings of specific games or a rival dev going out of his/her way to try to bring their own game back into the spotlight is as of yet unknown but Microsoft doesn't plan on doing anything about it. Maybe ever.

It's sad.

Edited by ProfessorEss

Y'know I'd put a lot more blame on Metacritic if I didn't think the whole reviewing process was kinda jacked from top to bottom. 
 
If reviews are written totally subjectively (because apparently even a modicum of objectivity has been deemed and excepted as impossible) and require a number score even though every reviewer out there seems to be against them, then what's the use of any of it to begin with? 
 
And why do they require number scores? That's my big question, and Giant Bomb is a great example. If Giant Bomb truly doesn't believe scores are important (possibly even detrimental), or that their five point scale doesn't apply to standard mathematical conversions then why is there a score at all? If the stars truly represent what they say then why not simply rate the game with a qualifying statement instead of something that can be so clearly translated into a number? 
  

@benjaebe: I do think Sessler nails it here but he raises the same question. He says "we hate scores but, you know, we have to so..." but no one ever seems to explain why they have to. I realize people have bosses, companies have policies, by how can I accept a reviewer who hates scores as having any integrity if they're putting a score that they don't believe in on every piece that they do?
 
As long as (pretty much) every reviewer out there is playing right into Metacritic's hands by sending them numbers that they know full well are going to be mathematically converted to a 100 point scale I can't see how we can place all the blame squarely on Metacritic. 
 
...oh and user reviews, man I can't understand how any pub, dev, or customer would be bothered with that bag of shit. The fact that Bastion's score could drop so drastically so fast is disturbing, but the fact that a Twiitter from Kasavin could raise it back up so quickly is almost just as disturbing.

Posted by wickedsc3

@RobertOrri said:

@Mordukai said:

Developers know Metacritic is crap. We know Metacritic is crap. Now someone please let the suits in the publishing house know about that.

Yes.

I'm sure they know its crap. But a bad review is a bad review, no matter where it comes from you don't want your game getting negative reviews.

Posted by DrRandle

Metacritc and numerical critical scores in general are a disease to this and every industry. There is no true value to any of these systems. It is all ludicrous. You cannot truly tell the quality of the game with a random number grasped from the air. Reading any number of user reviews on Apps on Amazon's App Store, for example, shows why people should not be allowed to just throw scores into the air. It's terrible, and nobody is providing any real qualitative statements to back it up.

Nobody gives numerical scores when they critique Van Gough. Nobody assigns an arbitrary number of stars to Mozart. Thumbs do not apply to the works of shakespeare. So if we are truly going to examine and critique video games, in order to both learn how to make better ones, and in form people what is good and bad about each one, then we need to move away from this system of instant gratification.

Posted by PrivateIronTFU

Metacritic is not the problem. The game developers and publishers (and users) who take Metacritic so seriously are the problem. If you want to hate something, hate them.

Posted by cikame

I hate metacritic.

Posted by Claude

I use metacritic. I like having a bunch of reviews to choose from and read. The user reviews and their overall score I take with a grain of salt. But I like bottom feeding too.

I'm not sure if metacritic fixed this, but I remember users spamming low scores before a game was even released a while back.

Posted by Rem45

Great piece Patrick! And Metacritic should really review its policy for allowing 'random' reviews in an attempt to grief certain game developers.