Power to the People, Until the Power's Out of Control

@Delta_Ass: it was from the futureshop website, but yeah the FF symbol is their logo.

@Delta_Ass: Seinfeld. Nice.

Metacritic is the most useless, idiotic concept that has plagued the entire industry. Developers need to stop giving a flying fuck about it, and publishers need to learn that Metacritic scores mean shit all for sales numbers. It is completely, utterly useless.

@White_Silhouette: Did you ever see the Amazon page for the Star Wars BluRay set? That had 1.5-2 out of 5 stars for weeks up until release and that is a site that even requires text with the scores. All this from people who had never seen or touched the set, but were going off of clips or information they had heard that they were unhappy about. People "reviewing" products yet to be released because they are unhappy about what's been shown off or have an agenda to push has been going on for awhile and probably will continue.

Interesting. I always wondered why some awesome games received oddly low scores on metacritic.

Amazon gets review bombed just as frequently. Recently I noticed a ton of negative reviews for Deus Ex Human Revolution based solely on the point that it requires a Steam install. It's a valid point of criticism, but not enough to justify an instant 1 star review.

Other games get lambasted for their DRM.

There's also the fact that people who work in the industry have been caught voting down competing games on metacritic while voting up their own. There was a thread on GAF a few months ago, where a few metacritic reviewers were giving super high scores for Bioware RPGs and super low scores to... I think it was the new Witcher game. Turns out when you look up their username, it brought you to a linkedin account showing they worked for EA or something like that.

I wouldn't assume these rushes of negative reviews are all from fans. Do some research, you may actually find something worse going on.

Reminds me of when Bioware was blaming DA2's low Metacritic score on /v/. Because it's impossible that, you know, that game was just a fucking awful yaoi fanwank.

But that's besides the point. lol, metacritic.

Review scores are very unreliable to the point where I do not bother with Metacritic or any other large gaming site like gamespot or IGN. I just use my own initiative now and it has served me pretty well so far. I still read reviews to see what the reviewer has to say and at times I see the reviewer mentioning plenty of good things about the game but the game would get a 7/10 for example. If a game looks good, I buy it, if it looks bad, I don't bother. People need to think for themselves instead of relying on review scores to decide on whether to buy a game or not.

The sad fact is that user reviews on the internet should not be using such a stratified scale for anonymous ratings. The larger the scale, the more weight a 0/10 or 10/10 vote bombing will skew results by mean average or even median scores. Most users don't understand or differentiate the ten point scale with 0.5 or 0.1 increment on established sites like IGN or Gamespot had; why would we allow a ten point scale for anonymous submitters. An unrealistic number of ratings are always on the extremes on Amazon, Yelp, Rotten Tomatoes as well. This is not unique to Metacritic, except ratings at Metacritic get tied to bonuses and financial progressions for unfortunate reasons. A thumbs up, thumbs down vote or 5 point scale would help, but not solve the issue. Personally I have ignored user ratings on Metacritic and would not display that user score on the games page or in search results without a click-through to a user reviews page.

@Vinny_Says: The only reason they care is because consumers review those sources. If consumers didn't care then developers would not care.

Great write-up, and it'll hopefully add a reason for publishers to not depend so heavily on the site.

Metacritic's interface is ass. Just use Gamerankings. Problem solved.

@krazy_kyle said:

Review scores are very unreliable to the point where I do not bother with Metacritic or any other large gaming site like gamespot or IGN. I just use my own initiative now and it has served me pretty well so far. I still read reviews to see what the reviewer has to say and at times I see the reviewer mentioning plenty of good things about the game but the game would get a 7/10 for example. If a game looks good, I buy it, if it looks bad, I don't bother. People need to think for themselves instead of relying on review scores to decide on whether to buy a game or not.

That's what I usually do. I can't understand why games that look amazing get horrible reviews because of stuff like "Oh, the story is boring."....I can understand if a story is boring, but if it's not a story driven game it shouldn't warrant a 6 or 7 because of it.

User reviews on those sites are nothing but spam. The vast majority of them are like this: (CoD: Black Ops) "COD SUX ITS LAME NOT REAL GAMING TOTALY BROKN. 1/10" I don't care who you are, a good game is a good game even if you're not a fan. The regular Metacritic score is a little better. At least they somewhat mask the hyperbole. Sometimes.

When taken in the right context, Metacritic is really useful. I can see that "Gee, Brad's review of Metroid Other M is quite a bit higher than what most people think and the complaints others have concern me. Maybe I'll GameFly it before buying it." It gives me an idea of what to expect from a game rather than blindly buying it. I don't put too much stock in the ratings anymore, but it often helps me sift the amazing games from the merely good ones so I can prioritize. Anything below the 70's is not worth playing ever.

Reviews don't mean a damn thing compared to word of mouth.

This same thing happened to Dragon Age 2. That game has legitimate issues, but there was a huge bombing of negative scores as the swell of frustration about so many pre-order bonuses occurred.

Proves why Metacritic is a BAD IDEA.

Metacritic is a reductive blunt instrument. Find writers you know and trust instead of using silly aggregators.

Man, this is fucking stupid.

@Vinny_Says said:

It's sad devs put so much importance on metacritic, even sadder when they care about user reviews where anyone can write anything

It's sad that devs have to put so much importance on Metacritic because their jobs, amount of pay, and future opportunities depend on the publishers that put so much importance on Metacritic because they need a collection of subjective numbers to tell them if something was a success or not.

They said how to fix it. Only allow a user rating if they have written a text review.

@dox said:

@Vinny_Says: The only reason they care is because consumers review those sources. If consumers didn't care then developers would not care.

Citation needed.

Fuck metacritic. I hate aggregated review sites... Worst fucking thing.

This is why Metacritic shouldn't allow user reviews at all, especially before the game is even released to the public.

Developers know Metacritic is crap. We know Metacritic is crap. Now someone please let the suits in the publishing house know about that.

User scores are just a little bit more useless than critic scores. But we already knew that.

The joke is that the only people that actually get paid more if they receive better reviews are the devs.

@Undeadpool: To be fair, regarding Dragon Age 2, it was the fans voicing their frustration at what they saw as bought reviews. It is the general consensus these days that Dragon Age 2 is not a great game, or even that horrible, rather being merely mediocre, but at the time the fans were seeing 94 & "Game of the decade!" reviews pop-up on Metacritic. Either reviewers were getting a very different experience than the fans, or something happened.

Then it was revealed that members of BioWare's team were stealth-posting 10 reviews on Metacritic. That, in addition to the horrible reaction BioWare had to the backlash, made Metacritic user reviews one battleground of many between the upset fans and BioWare.

I knew publishers took MetaCritic very seriously, but I didn't think that meant they took the USER reviews seriously too.

@Mordukai said:

Developers know Metacritic is crap. We know Metacritic is crap. Now someone please let the suits in the publishing house know about that.

Yes.

Metacritic can die in a fire. It's nothing but problems to the business.

Great piece Patrick! And Metacritic should really review its policy for allowing 'random' reviews in an attempt to grief certain game developers.

I use metacritic. I like having a bunch of reviews to choose from and read. The user reviews and their overall score I take with a grain of salt. But I like bottom feeding too.

I'm not sure if metacritic fixed this, but I remember users spamming low scores before a game was even released a while back.

I hate metacritic.

Metacritic is not the problem. The game developers and publishers (and users) who take Metacritic so seriously are the problem. If you want to hate something, hate them.

Metacritc and numerical critical scores in general are a disease to this and every industry. There is no true value to any of these systems. It is all ludicrous. You cannot truly tell the quality of the game with a random number grasped from the air. Reading any number of user reviews on Apps on Amazon's App Store, for example, shows why people should not be allowed to just throw scores into the air. It's terrible, and nobody is providing any real qualitative statements to back it up.

Nobody gives numerical scores when they critique Van Gough. Nobody assigns an arbitrary number of stars to Mozart. Thumbs do not apply to the works of shakespeare. So if we are truly going to examine and critique video games, in order to both learn how to make better ones, and in form people what is good and bad about each one, then we need to move away from this system of instant gratification.

@RobertOrri said:

@Mordukai said:

Developers know Metacritic is crap. We know Metacritic is crap. Now someone please let the suits in the publishing house know about that.

Yes.

I'm sure they know its crap. But a bad review is a bad review, no matter where it comes from you don't want your game getting negative reviews.

Y'know I'd put a lot more blame on Metacritic if I didn't think the whole reviewing process was kinda jacked from top to bottom. 
 
If reviews are written totally subjectively (because apparently even a modicum of objectivity has been deemed and excepted as impossible) and require a number score even though every reviewer out there seems to be against them, then what's the use of any of it to begin with? 
 
And why do they require number scores? That's my big question, and Giant Bomb is a great example. If Giant Bomb truly doesn't believe scores are important (possibly even detrimental), or that their five point scale doesn't apply to standard mathematical conversions then why is there a score at all? If the stars truly represent what they say then why not simply rate the game with a qualifying statement instead of something that can be so clearly translated into a number? 
  

@benjaebe: I do think Sessler nails it here but he raises the same question. He says "we hate scores but, you know, we have to so..." but no one ever seems to explain why they have to. I realize people have bosses, companies have policies, by how can I accept a reviewer who hates scores as having any integrity if they're putting a score that they don't believe in on every piece that they do?
 
As long as (pretty much) every reviewer out there is playing right into Metacritic's hands by sending them numbers that they know full well are going to be mathematically converted to a 100 point scale I can't see how we can place all the blame squarely on Metacritic. 
 
...oh and user reviews, man I can't understand how any pub, dev, or customer would be bothered with that bag of shit. The fact that Bastion's score could drop so drastically so fast is disturbing, but the fact that a Twiitter from Kasavin could raise it back up so quickly is almost just as disturbing.

This exact thing is a pretty huge problem in the XBLI section on XBL as well. It seems like any game that reaches the top 60 gets a surge of 1 star ratings almost without fail. Whether this is the work of a small group of individuals trying to bring down the ratings of specific games or a rival dev going out of his/her way to try to bring their own game back into the spotlight is as of yet unknown but Microsoft doesn't plan on doing anything about it. Maybe ever.

It's sad.

@Vegsen said:

Proves why Metacritic is a BAD IDEA.

Metacritic is not a bad idea, its actually a great idea, hows its used is the problem. How can having a large number of different reviews all in one place be a bad idea? The fact that bonuses are paid based on Metacritic scores and the bad policing of spam reviews are the problems, not Metacritic itself.

All reviews should require text. Just picking a number isn't sufficient enough. All reviewers should also have their reviews approved for a period of time to make sure they're not spam or fanboys.

@Delta_Ass said:

"Excuse me I'd like to give a user review of 1.0."

"Certainly. May I ask why?"

"For spite."

"Spite?"

"That's right. I don't care for the developer."

"I don't think you can review a game for spite."

"What do you mean?"

"Well if there was some problem with the game, if it were unsatisfactory in some way, then we could do it for you, but I'm afraid spite doesn't fit into any of our conditions for a review."

"That's ridiculous, I want to review it. What's the difference what the reason is?"

You can't review a game based purely on spite."

"Well so fine then... then I don't like it and then that's why I'm giving it a 1."

"Well you already said spite so..."

"But I changed my mind..."

"No... You said spite... Too late."

This is pretty much it. Or it's 4chan morons "doing it for the lulz".

@JoeyRavn said:

What worries me most is that if I sincerely dislike a game (for example, Bastion) and give it a low score, my rating will be dismissed as "spam". What an awesome tool Metacritic is.

Either you fucking love a game, or you're just a butthurt loser spammer.

Metacritic has never been good for user reviews. It seems like only the worst of the youtube commenters go there and post even more bullshit. Why are publishers unaware of that?

Who the fuck buys games based on their metacritic USER reviews?

@Paul_Is_Drunk: The issue then becomes (beyond "two wrongs don't make a right") what did these people actually hope to accomplish? I remember hearing about the "stealth reviews" and seeing the 0/10 user reviews popping up at around the same time, so I'm not sure you can completely say that one was in reaction to another. I actually think DAII turned out fine (better than Origins in some ways, worse in others), it was just weird seeing THAT many people get THAT dedicated to absolutely ruining this game and then have it amount to nothing. DAII outsold Origins (which is merely a statement of fact and not a measure of quality) and yeah, most people have now just kind of shrugged their shoulders and agreed that it wasn't that great.

@ProfessorEss said:

Y'know I'd put a lot more blame on Metacritic if I didn't think the whole reviewing process was kinda jacked from top to bottom.

If reviews are written totally subjectively (because apparently even a modicum of objectivity has been deemed and excepted as impossible) and require a number score even though every reviewer out there seems to be against them, then what's the use of any of it to begin with?

And why do they require number scores? That's my big question, and Giant Bomb is a great example. If Giant Bomb truly doesn't believe scores are important (possibly even detrimental), or that their five point scale doesn't apply to standard mathematical conversions then why is there a score at all? If the stars truly represent what they say then why not simply rate the game with a qualifying statement instead of something that can be so clearly translated into a number?


@benjaebe: I do think Sessler nails it here but he raises the same question. He says "we hate scores but, you know, we have to so..." but no one ever seems to explain why they have to. I realize people have bosses, companies have policies, by how can I accept a reviewer who hates scores as having any integrity if they're putting a score that they don't believe in on every piece that they do? I'm not singling out Giant Bomb or Sessler but they're good examples because they have some clout to try and change this if they really believed this.

As long as (pretty much) every reviewer out there is playing right into Metacritic's hands by sending them numbers that they know full well are going to be mathematically converted to a 100 point scale I can't see how we can place all the blame squarely on Metacritic.
...oh and user reviews, man I can't understand how any pub, dev, or customer would be bothered with that bag of shit.

Really, you don't understand why scores are needed? Its because people are fucking lazy. They don't want to read a review, they just want to see a number and make their decision based on that.

@Meatsim: Agreed. A written review should be required. I also they should be moderated to some degree before they're included in the actual user score.

@RobertOrri said:

@Mordukai said:

Developers know Metacritic is crap. We know Metacritic is crap. Now someone please let the suits in the publishing house know about that.

Yes.

+1

@BisonHero said:

@Delta_Ass said:

"Excuse me I'd like to give a user review of 1.0."

"Certainly. May I ask why?"

"For spite."

"Spite?"

"That's right. I don't care for the developer."

"I don't think you can review a game for spite."

"What do you mean?"

"Well if there was some problem with the game, if it were unsatisfactory in some way, then we could do it for you, but I'm afraid spite doesn't fit into any of our conditions for a review."

"That's ridiculous, I want to review it. What's the difference what the reason is?"

You can't review a game based purely on spite."

"Well so fine then... then I don't like it and then that's why I'm giving it a 1."

"Well you already said spite so..."

"But I changed my mind..."

"No... You said spite... Too late."

This is pretty much it. Or it's 4chan morons "doing it for the lulz".

"I don't think the game's any good."

"You're just a butthurt spammer! Troll! Everything's amazing and nothing is bad!"

"Except this. I wasted money on this game."

"If it's bad, why not write a ten-thousand paragraph review about it, huh?!"

"Because I've wasted enough time on this game already? And who reads these reviews, anyway? It's not like I'm getting paid for this shit."

"Troll! Spam! Lulz! Butthurt! BBS Door Games!"

Metacritc is not crap, publishers deciding bonuses etc off Metacritc score is crap.

I think the oficial Metacritic scores are fine( they do represent the majority of profesional reviewers after all),

but the user ones are bullshit. Whoever says not to listen to the profesional scores and just go by user ratings are idiots.