Log in or sign up to comment
139 Comments
  • 139 results
  • 1
  • 2
  • 3
Posted by patrickklepek
When things go sideways on Metacritic, developers don't have many options for appeal.

Supergiant Games and Signal Studios were in awkward positions earlier this month, after the average user review score for each game plummeted overnight and seemingly without justification.

It was unclear who was responsible for the sudden change, and there are limited tools for those affected that don't come across as...weird.

The developers of Bastion and Toy Soldiers: Cold War asked fans to submit reviews to Metacritic, which resulted in a slight bump for each but an average score ultimately much lower than it was previously.

After speaking to Supergiant and Signal, I ran a story earlier this week about the challenges the studios faced in addressing these issues.

Metacritic has since identified the users responsible, both banning the users themselves and removing their scores putting things out of balance.

The users were targeting certain games, and hitting them with zero scores without a written review, raising eyebrows.

"A publisher of a different game let us know that he noticed that his game had been 'bombed' by a bunch of zero ratings in an unusually short period of time," said Metacritic co-founder Marc Doyle in an email to me today. "While investigating those ratings, we noticed a group of user accounts and activities that were clearly illegitimate and violated our terms of use."

Doyle's team found a group of users had been coordinating similar attacks on other games within Metacritic's database, allowing Metacritic to fix the "bombings" that had been targeted at other games.

Metacritic is often used to determine royalty and bonus payouts for developers, though its exact use varies from publisher to publisher. I've never heard of a publisher leveraging user reviews as a metric for payment because of situations like this, but especially for small studios, perception is king, which is why user reviews remain very important.

Signal Studios president and creative director Douglas Robert Albright III had proposed a fix to the problem.

"The way to fix Metacritic user reviews is to simply require a written review and verify user accounts," he said. "If it was just some random blog I'd say whatever. But this is a major news review aggregator that should have better oversight and some standards."

Staff
Posted by wemibelec90

Glad to hear someone is cracking down on these jerks.
Edited by ZackHoagie

So... why exactly were they doing this? I mean, trolling is pretty unreasonable by design, but usually trolls have some sort of selection process for their targets.

Posted by elcalavera

Very nice.

Posted by BestUsernameEver

@elcalavera said:

Very nice.

Is it?

Posted by BestUsernameEver

@ZackHoagie said:

So... why exactly were they doing this? I mean, trolling is pretty unreasonable by design, but usually trolls have some sort of selection process for their targets.

Yeah, metacritic is the target.

Posted by prestonhedges

They should just ban all user reviews below 100%. Problem solved.

Posted by Tesla

Well, good on Metacritic for actually doing something about this.

Posted by MayorFeedback

Busted.
Posted by CaLe

Why would they have targeted Bastion? Who did it ever hurt?

Posted by Largo6661

see this is why metacritic is bullshit. its beyond me why anyone puts any kind of value on whats posted on that abomination of a site. hope it dies in a fire!

Posted by Giantstalker

You'd think this would happen every day, considering the barrier for entry is so minimal.

Posted by SomeJerk

Dear Industry,

Stop taking user reviews seriously, and start taking reviews seriously. If they all say the game is shit but it sells like nude shots of the latest idol, fix it. If they all think the game is great but it sells only a few copies, learn to market properly. Holy shit.

Posted by DystopiaX

@Largo6661 said:

see this is why metacritic is bullshit. its beyond me why anyone puts any kind of value on whats posted on that abomination of a site. hope it dies in a fire!

Most devs only look at staff scores, not user ones, so that doesn't matter. It would be like saying Giantbomb should get the axe cause you disagree with the user reviews on the site.

Posted by laserbolts

Fuck metacritic.

Posted by SinisterInfant

If I were to put on my conspiracy theorists hat, It could be publishers themselves posting bad reviews to drive down the metacritic score. Both of those games are now several weeks past their initial release window. From a publisher view point the majority of sales have likely been made. If they owe money to the developer based on a score, they could drive the score down pay the developer for the lower score then delete their comments and restore the score. Seems like an easy way to anonymously save some funds.

Posted by falling_fast

while they're at it they can ban all the critics who are giving Gears of War 3 less than perfect scores *smirk*

Posted by TheOtherSuperMarioBro

I definitely approve. Metacritic user reviews are some of the biggest shams on the internet

Posted by DystopiaX

@SinisterInfant said:

If I were to put on my conspiracy theorists hat, It could be publishers themselves posting bad reviews to drive down the metacritic score. Both of those games are now several weeks past their initial release window. From a publisher view point the majority of sales have likely been made. If they owe money to the developer based on a score, they could drive the score down pay the developer for the lower score then delete their comments and restore the score. Seems like an easy way to anonymously save some funds.

Would be dumb, plenty of people still buy games a couple weeks after they're out, and payouts are based more on critic than user reviews.

Posted by Baconbot

@ZackHoagie: Stupidity probably.

Good news to hear and good on Metacritic for cracking down.

Posted by autobzooty

metacritic slowly realizing that the service it provides makes no sense.

Posted by Cynic04

WTF, why would they go after the little developers? Not a good way to earn sympathy.

Posted by Slaker117
@ZackHoagie said:

So... why exactly were they doing this? I mean, trolling is pretty unreasonable by design, but usually trolls have some sort of selection process for their targets.

Well, both games were Xbox Live titles that aren't on PSN. I'd like to believe platform fanboyism is dead, but that seems like a possible motive in this case.
Posted by vr00m

Why would they count outliers anyway, especially ones with no written review.

Posted by Honk

It's absolutely awesome that someone is cracking down on these shenanigans. Unacceptable behavior like this can't be allowed.

Posted by ZackHoagie

@Baconbot: The weird thing is though is that they specifically targeted two Summer of Arcade games instead of say, Resistance 3 or Hard Reset or Gears 3 or what have you. As shitty and console-warsish as it sounds, I have a pretty good feeling this might of been Sony fans.

Posted by PXAbstraction

For once, Metacritic does something useful.

Posted by ZackHoagie

@Slaker117: Dude, look at these comments at a story about Sony's PSN woes or what have you and you'll realize console warriors are still fighting the stupid, stupid, stupid fucking fight.

Posted by Fobwashed

I use metacritic and before them, gamerankings not to get an aggregate review but to get a list of reviews from sites in one place with blurbs. I never paid much mind to the user reviews though. I DO check and read user reviews for purchases I make on amazon, newegg and I look at the aggregate for movies on RottenTomatoes.

Good on them for actually doing something when they could just have easily have not cared. I wonder what their barrier to entry is for the critic review section.

Posted by Delta_Ass

@SinisterInfant said:

If I were to put on my conspiracy theorists hat, It could be publishers themselves posting bad reviews to drive down the metacritic score. Both of those games are now several weeks past their initial release window. From a publisher view point the majority of sales have likely been made. If they owe money to the developer based on a score, they could drive the score down pay the developer for the lower score then delete their comments and restore the score. Seems like an easy way to anonymously save some funds.

Publishers don't give bonuses based on user reviews. It's based on the aggregate review scores from professional publications. What you suggest is insane.

Posted by Jimbo

Cool, they should worry more about the 'Everything with an ad budget is amazinggg! 10/10' bullshit coming out of the reviewers. Also about their own incompetence when it comes to converting review scores, 5* scales converting 10% too high etc.

Posted by John1912

Yea WTF? I can see people being random dicks and give 0s. But a group of people actively working to do so? Why?

Edited by Troncek

Meta who?

I almost never use that site. If I'm interested in a game, I check out videos, interviews, articles and other stuff, so by the time the game is actually released, I have gathered more than enough information to make a decision.

I also don't read other people's reviews (including here on GB), because they usually don't tell me anything new or useful.

It's sad to see industry insiders rely so much and put weight on this stuff.

Posted by lightsoda

Good on them.

Posted by Baconbot

@ZackHoagie: Probably true.

Posted by SSully

I wonder if they banned me. I made an account and scored all the games as 10's who got hit by these guys.

Online
Posted by JackG100

What kind of tards think that internetpolls give a fair picture of reality anyway?

Posted by Vexxan

Great to see something was done about it.

Posted by eshchan

@patrickklepek Patrick, are you sure developers of Bastion also asked its fans to write positive review on Metacritic? I know PR for Signal Studios did it but I thought SuperGiantGames did not.

Posted by DonutFever

I think it's weird that small teams were being targeted.

Posted by NinjaCommando

@ZackHoagie: trolls dont have a target they just want any type of response...

Posted by Corvak

User reviews make no sense. At least, it makes no sense for them to be given equal stature as a professional review. Professional reviewers typically get their jobs through good writing practices, and for being a fair judge of a game, though it is, of course, still the reviewer's opinion of the game. This is no different than major online storefronts like Amazon - who hire people to track down and remove manufacturers or publishers leaving fake reviews praising their products, or attacking a competing brand.

Because reviews are opinion pieces, just reading one site isn't the answer either, which is why Metacritic exists. Because so many of us, both gamers and industry people wrongly feel that the score is the be-all end-all of every review, Metacritic's aggregate score becomes the most accurate way to gauge the reception of a game. Sadly, we cannot break down video games into objective hard numbers to compare them - it just doesn't work - the "fun" of a video game isn't something that can be measured by a machine.

Personally, I like that they line up all the professional reviews for me to read, instead of having to dig through pages of ads and clutter. Giant Bomb is probably the best laid out gaming news site i've found - The others tend to be crammed with as much content as humanly possible onto the front page, or set up like a blog, where everything older than six hours has been pushed off the bottom.

If we start ignoring user scores, Metacritic or sites like it can be a way for quality releases like Bastion to get noticed without spending millions on mass marketing. If we continue to be distracted by numbers, it'll just be another battle in the pointless console war, and that's a war the trolls and moronic fanboys always win.

Posted by StormHarbinger

Great job Patrick, it's hard not to see your article and exposure as having an impact on Metacritic's actions. I'd love to think they would correct the problem on their own, but...

Anyway, kudos to you for making an impact time and again.

Edited by phrosnite

That's why I've always said that User reviews or overall users' score meant nothing.

Edited by Rapid

Why, Why hurt developers??? If they were trying to make a statement couldn't they at least have actively rated 100?

Posted by Klei

I've never cared for User Reviews before anyways and I will never. Most guys who writes scores played the game for 30 minutes. heck, that is, if they even played it. They usually overhype it or bitch it to death. It's pretty rare to find level-headed reviews.
Edited by mordukai

Too little, too late. This feels like a "sweeping it under the rug" type of solution to this situation. The damage is already done snd there has to be a long term solution that will prevent this from happening in the first place. If it wasn't for those developers and this article pointing out the problem then Metacritic would have done nothing at all. They simply don't care. Metacritic needs a major overhaul and they reviews are taken. As an idea Metacritic is great but it was executed poorly. Metacritic must realise that it's playing with the livelihood of people. I hope this case won't get swept under the rug because this issue must be addressed.

Posted by Largo6661

@DystopiaX: i have no problem with review scores or peoples opinions on a game, What i have a problem with is what metacritic has done to gaming culture. I mean 5 years ago 7 was a good score now a 7 is a terrible score, and devs say things like "hope this scores at least an 8 on metacritic or we are done". I hate the way a games success often lives and dies on its metacritic score. At the end of the day its just an arbitrary number. My main problem with the site is how it down plays the importance of the words writen in the review and pretty much focuses in on a number to convey to a person whether a game is good or not, and then whats next? metacritis scores for reviewers or maybe specific devs? its just a dumb site. its the review itself thats important not the number added at the end.

Posted by Boiglenoight

Good for Metacritic.

Posted by heatDrive88

Who's to say that the people who are running these illegitimate accounts purposely dropping the Metacritic score aren't the same people who will be fiscally responsible for handing out bonuses to the developers, as a method of cutting the cost of that paying out that bonus to developers?

  • 139 results
  • 1
  • 2
  • 3