@LiquidSwords: Indeed, I have never, ever cared what Rotten Tomatoes has to say and the only reason I know of Metacritic's existence is because they talk about on GB; I have never even been on Metacritic's site.

If something is too popular or unpopular, I usually find that feel the opposite way about it.

When will companies realize that Metacritic ,is freaking BROKEN!?

Good job Patrick

I have never understood how devs and publishers use Metacritic to determine bonuses and what not You would think sales would be the driving factor. I would not give on F if my game on metacritic was a 24 but I sold 300 million copies casue I'd be rich

great article. This is a really shameful process. Metacritic should be more proactive about tracking down the source of these negative reviews.

I wouldn't be surprised to see competition for these publishers getting on metacritic and rating their games down. This kind of shit is going to make them irrelevant.

Metacritic determines bonuses and royalty payouts for many.

I wonder how many Metacritic accounts some devs have.

@SpudBug said:

great article. This is a really shameful process. Metacritic should be more proactive about tracking down the source of these negative reviews.

They need a synergy based paradigm, obviously.

@TheHBK said:

That game sucks ass anyway.

Dude you are so edgy I cut myself on your post.

I think requiring justification in the score through actual text would be the best thing. What people don't think about as much (as it more affects consumers rather than developers) is the possibility of users spamming games with overly positive reviews, either to cause mischief or express their fanboydom. Of course developers wouldn't care, but I think that it would give consumers looking to purchase the game an inflated sense of how good it is perhaps. Just a view from another perspective on the same issue.

So what, there are as many positively biased user reviews as negative ones anyway. Think the user ratings as some sort of 'hype' meter then it all makes sense. The critics ratings number actually is a fairly accurate representation on the game's reception on the media circles, no wonder the publisher and developers trust them as a source alongside million other figures. I think Metacritic is no less an authoritative source on a game's quality than any other single games 'review' sites, because it is simply a representation of a glut of them.

Fanboys and trolls from /v. That is all.

Thank you for drawing attention to this abuse Patrick.  I hope it lights a fire under CBS Interactive to take these issues seriously.  And they are serious, despite what many of my fellow commenters seem to think.
 
Gaming is big business.  Big businesses require lots of people who know specialized skills: finance specialists, marketers, managers; HR folks.  It'd be great if all of those people were gamers as well, but that's not realistic.  How many people in the recording business don't know any Beethoven aside from the opening of the fifth symphony?  How many people in the publishing business have never read Hemingway?  How many people in the film business have never seen Bergman or Fellini?  These people don't make artistic decisions.  They make business decisions.  Their shareholders demand it.  And sites like Metacritic are the closest things to impartial opinions they've got.
 
And both scores matter.  Some people distrust critics and want the opinions of buyers.

I love how the fix is simple, make the person write a word review, but does the site do it, "No we dont want to upset our users" what kind of excuse is that? Hell a simple sentence of "I hate this game it sucks" is better than some asshole 4chan wanna be trolls rating random games 0 "for the lutz!"

here's my thing. If you are only basing your opinion on a game, based on and arbitrary score given by other people, then maybe you shouldn't be playing to begin with. I for one, know that if I played based squarely on that, I would have missed a lot of good games. Just because someone gave something a bad review or score, doesn't mean you won't like it. Maybe you should go out there, try it for yourself, and then, I know, this is shocking, make your own opinion. Just saying.

sounds like issues I've had with credit reporting agencies. yeah they suck but you don't get a choice whether you use them or not, and their information is always inaccurate. I don't think this deserves a news article. game devs just need to police metacritic. paying for reviews would be one way to do that.

actually, just cut right to it and PAY METACRITIC for good reviews.

It sucks that it seems to be the norm to grief anyone, anywhere, anytime now and days. I hope they revise the system and take the issue into consideration.

This wouldn't be an issue if Metacritic took any responsibility at all for how their site works. How can you allow textless reviews if those reviews still count towards the average score? It just doesn't make sense.

I know that even if they did require written reviews there would be a lot of bullshit spam reviews, and the resources required to filter every user review on a regular basis would be significant, but that's the only solution I see working.

did that guy just refer to GiantBomb as "some random blog" ???

What's that? It's a system where anonymous "this game sucks" reviews can cut into your pay, and you don't know who to look at first? I'll give you a hint: He pays the check that gets smaller.

@benjaebe: I didn't mean to sound like I was calling them out, and I should've said "making scores available" as opposed to "sending" - but I stand by my point that if you hate numeric scores, and you hate Metacritic using the scores you give games maybe consider not giving them scores. Or at least give us an explanation as to why you (especially reviewers with clout) have to give numeric scores.

haters gonna hate

Who pays for these kind of "services"? Or is it just trolling? So much shady shit goes on it's hard to tell sometimes. Great article, interesting problem.

Another reason why I wish some people were just outright banned from the internet...

Why would anyone ever go by user scores? Most people just seem to have a habit of loving something and giving it a 10 or not liking it and just giving it a 1.

Recently I've only read reviews from Giant Bomb. I believe that one of Jeff's main goals of the website was to reestablish the byline, and for me he and the crew have succeeded in doing this. I know their tastes, and I know when to trust a review and when to disregard it and buy the game anyway. And for me, that is the most important thing about any kind of critical journalism (if that's a thing).

"The way to fix Metacritic user reviews is to simply require a written review and verify user accounts," said Albright. "If it was just some random blog I'd say whatever. But this is a major news review aggregator that should have better oversite and some standards."

I completely agree with this. I'm not big on scores, but I understand the reason Developers like Metacritic. They want to be able to judge what the public perception of the game is based solely on individual experiences. Metacritic helps funnel that into one place.

This is sound, in theory.

Metacritic should know by now how useful they are to developers and should have things in place that keep these kinds of things from happening.

That being said, I really have problems with the idea that Metacritic affects bonuses/pay. Again I get why it's useful from a development standpoint. It can really give you a sense of how much individual players (no matter how small majority the people who actual write them may actually be) enjoy or dislike your game and sometimes even why. However, this should determine how Developers make games in the future, I don't like the idea that this information can affect what they get paid. If they put in the work, they should still get paid. Sales will often determine what you're worth as a developer- although I suppose I have no defense to the argument that sales don't mean those who bought the game actually thought it was good, but on the same front I believe whether or not they buy another your bonuses or extra pay of whatever kind should come from this game not the possibility of better made next games.

I don't know, just my thoughts. Apparently I have a lot of them on this topic so I'll just leave it at that.

Interesting article.

I have never used metacritic to help me decide if I want to buy a game, but I do like looking at it after a game has been out for about a month to see where it finally lands. If I want to know if a game is any good, I come here.

Misleading title. It has nothing to do with "the people". If you put stock in shit that some website most people never even visit says (based on completely random visitors rating things) to the point that you base salaries and bonuses and careers and layoffs on it, then that's on you not on the people.

Fuck. FUCK. This is genuinely disheartening. I can't fathom intentionally attempting to obliterate any person's livelihood, but when they're trying their damnedest to bring innovation, ingenuity, and interest to an increasingly stale market? Really? Are these scumbag spammers trying to express disdain for indie projects? Are they truly just fucking bored? I wish I could go buy these games immediately to show some meager support (I would've already bought Bastion), but I unfortunately don't own a 360. Fuck this whole mess of a situation, man.

The efficiency of metacritic, with its glanceable, if shallow, review consolidation if an enticing service that offers a truly useful service for those of us with little time in the day to digest full reviews, however, it is a shame that such erroneous errors can arise with such services. Circumstances that can have such a negative, monetary effects on game developers and must be remedied.

I almost never solely rely on critic scores when it comes to anything. I have loved not so good buggy games and hated critically praised ones as well. I will look at scores and use them to sway, but if I am interested in something 100% nothing is going to stop me. As fo user reviews...I NEVER look at those unless they are on here.

I still can't believe bonuses are awarded based on anything but sales, as that's all the publisher should care about: how much money did they get. Sales should speak for themselves.

@RedSox8933 said:

Recently I've only read reviews from Giant Bomb. I believe that one of Jeff's main goals of the website was to reestablish the byline, and for me he and the crew have succeeded in doing this. I know their tastes, and I know when to trust a review and when to disregard it and buy the game anyway. And for me, that is the most important thing about any kind of critical journalism (if that's a thing).

Yup, once you realize what this site is about and have a feel for the staff you're crazy to go to a random website and trust it. GB is really the only place I trust anymore. Watching Jeff's at-home video the other day pretty much reminded me why I trust the site, he sort of touches on the subject of things like this in the video and I loved it.

@Dark_Lord_Spam said:

Fuck. FUCK. This is genuinely disheartening. I can't fathom intentionally attempting to obliterate any person's livelihood, but when they're trying their damnedest to bring innovation, ingenuity, and interest to an increasingly stale market? Really? Are these scumbag spammers trying to express disdain for indie projects? Are they truly just fucking bored? I wish I could go buy these games immediately to show some meager support (I would've already bought Bastion), but I unfortunately don't own a 360. Fuck this whole mess of a situation, man.

I agree, it's extremely frustrating, especially when I've seen into the dev as much as we have of Bastion and you realize there's tons of other small companies out there like them trying to make a living and people just shit on it for no reason other than it's funny or they're bored, like you pointed out.

Also, Bastion is available on Steam for next to nothing, buy it and support them please.

The toy soldiers QL looked shitty, the TNT was boring with people begging you to play more Age of Empires, and I didn't agree with the review. I think Metacritic was right on the money and I trust them most of the time.

that sucks

Imagine if Jim Carrey's or George Clooney's or Spielberg's paycheck was affected by Metacritic, or Rotten Tomatoes, or way back to Siskel & Ebert?

How quickly would that break Hollywood? Clearly there is a correlation where more sales means more residuals, but once the movie is done, you get the percentage in your contract, or SAG gets on the studio's ass. And then you're on to the next job.

Remember these business fuckers with the metacritic carrot on a money stick, are the same fuckers with the "fast track to production" carrot for QA testers. And we know how that goes.

@Dark_Lord_Spam said:

Fuck. FUCK. This is genuinely disheartening. I can't fathom intentionally attempting to obliterate any person's livelihood, but when they're trying their damnedest to bring innovation, ingenuity, and interest to an increasingly stale market? Really? Are these scumbag spammers trying to express disdain for indie projects? Are they truly just fucking bored? I wish I could go buy these games immediately to show some meager support (I would've already bought Bastion), but I unfortunately don't own a 360. Fuck this whole mess of a situation, man.

I doubt Supergiant or anyone else is going to go bankrupt just because someone on a website clicked a button. This is just PR wankery at its finest.

People are f***ing stupids. I don't get why forums and boards have moderators, but Metacritic (or anyone else) user reviews do not. Although maybe Gamefly does actually moderate them; they seem to have great user reviews. Even the 1's (out of 10) usually have some text, even just a line or two, about why that person hated the game. With awful grammar, naturally. But not writing anything? You're just being a di*k and loser.

Metacritic needs to get on top of this sh*t. And it would be great if we could just beat up people with mental problems like this; anyone who does sh*t like this randomly is a bane on society. On the internet AND in real-life.

So how long before Activision starts downvoting their own games at launch to save from paying out bonuses to their developers incentivized bonuses based on Metacritic? Or have they been doing that already?

I was not aware of this!

This is why you can't trust user reviews for video games. There's too much stupid factor.

There are the 0 reviews and then the counter-0 reviews and then the counter-counter-0 reviews.

So at one point they were going to give away free codes for the game, if people wrote positive reviews for said game? Wouldn't most people writing those reviews have never actually played the game? I guess its the same as the bad reviews but still weird.

Hi Patrick. It is Collin with two l's

Your friend,

Collin

What's the point of giving people free codes to a game they already played.... unless<end>

Look, if you are going to a site, and basing your buying off of what score something got, then you have other problems that aren't going to be fixed here.

As someone else put, if you look at most sites, especially now about say MW3 or BF3 you will see this, "MW3 sucks ass, BF3 is better" or "BF3 sucks. MW3 is going to be amazing". Well, really people, all you do is have pissing contest.

Just because you don't like a game, doesn't mean it sucks. I am not a huge fan of GoW, but i will tell you right now, it's an amazing game. It's just not a game that is for me. does that take away from it being good at all? NO. Why? Because it wasn't made for ME. It was made for a lot of people, and I just happen to not like it much. But I have enough of a brain to know that just because I don't like something, it doesn't mean it sucks. It just means, I didn't like it.

On the filp side of that, there have been plenty of games that people have said suck, but that I have to say I have enjoyed. Just because someone comes on and gives a bad review, doesn't mean that you shouldn't play it. Make up your own mind, and form your own opinion.

As far as Amazon goes, listen people, if you go onto Amazon, and you give something a 1 star review, and your gripe is with the shipper or anything BESIDES THE ACTUAL PRODUCTS, then shut the fuck up. You are here to review a product, so review it. No one cares if it didn't get there in time to meet your standards. that in no way represents the product, that represents the shipper.

@bransonhuggins said:

...snip...

As far as Amazon goes, listen people, if you go onto Amazon, and you give something a 1 star review, and your gripe is with the shipper or anything BESIDES THE ACTUAL PRODUCTS, then shut the fuck up. You are here to review a product, so review it. No one cares if it didn't get there in time to meet your standards. that in no way represents the product, that represents the shipper.

Amen. I hate that. I'm an avid reader who's recently fallen in love with my new Kindle, so I'm reading user reviews on Amazon a lot lately. It seems like every fourth review is some twit who's pissed his book came a day late or some bull. You find the same thing on Newegg. Some nitwit fried his new motherboard or some other asinine thing and puts up a zero review.

The one good thing on Newegg, is I've actually often seen the manufacturers, or the Newegg people, replying to these reviews and finding out wtf is really up. I wish Amazon had that. Wouldn't be a solution for Metacritic, I don't think. It's just too big.

@shinyidol said:

Hi Patrick. It is Collin with two l's

Your friend,

Collin

http://blog.eu.playstation.com/author/cmoore/

Shit, he's right. Even the typos themselves are pointing it out when it happens.

@TadThuggish said:

The way to fix Metacritic is to never use Metacritic.

QFT.

The best solution for that site is for everyone to ignore it. It's deeply flawed and overly influential.

Why can't people do this to games that deserve it (more)?

Ryan & Vinny: "Now it's the real internet." ... "Hackers. Spammers. Dick-pill ads. Walgreens is having a special."

I think Steam should aggregate review scores from a different system.

I thought the internet jackasses had reached apogee, but I guess not.

Even with professional reviews, the mean score (based on an aggregation of dissimilar data points) is a fundamentally flawed metric. I'd love to see someone create a Metacritic competitor that allowed users to custom-aggregate scores based on sources they trust, whether those sources be professional games journalists or other users.