Log in or sign up to comment
61 Comments
  • 61 results
  • 1
  • 2
Edited by Sunjammer

I’m sort of a statistics idiot. I am endlessly fascinated by statistics, especially as a game developer, where they power everything from animation to scoring systems, but I also have a lot of love for what statistics can tell you about the world at a glance. It’s a comforting feeling to see reality described as a value.

Industry has long searched for reliable metrics on which to gauge success. The traditional metric is “how much did it sell and at what price versus our investment”, but this statistic is vividly dependent on time as a factor. How do you get a gauge of the success of a product when that product’s lifespan is considerable? The concept of the “indie darling” where a game turns out to be a success where little was expected is warm and fuzzy and fun to think about, but completely untenable as a reliable business model.

For analysts, prediction of success is a progressively sexy concept as investment increases. In video games, recently I was made aware of so-called “mock reviews”, a practice in which games journalists write game reviews for unfinished products for internal use by the publisher so as to predict how the game will score at final release, and, I assume, determine the marketing budget; as a game developer, how much real change can you introduce into a game at a point where the game is already “reviewable”?

Finally though, as a game is released, with the long tail of game sales these days, what determines success?

Unfortunately for everyone, consumer and industry, “success” is currently measured in the Metascore. It’s time for me to ramble.

Averages are wonderful in their need for reinterpretation. They are the most boring of statistics, existing only to get rudimentary ideas, remove edges, peaks and valleys. The average of a triangle’s vertices give you its midpoint; A representation of the triangle for certain, but what a pitiful representation. Such a representation only has value through interpretation and contextualization. Consider the average CPU use of an application. It might idle and do nothing, and it might burn every core you have, and so your average, out of context, is almost completely useless. You’ll be sat at that dull median, knowing even less than when you started. Averaging a system with lots of variation is, as far as I can tell, silly; The only thing you can measure is a tendency, and a tendency is not a precise value. The aggregate of a game would permanently be in the 80s or 70s. Only outlier games would diverge from this average

Video games media is about as score-driven as media comes. They are a natural fit for scoring, after all. In playing games, especially of yore, success is eminently quantifiable, and this quantification of reality and success is a big draw for a lot of gaming as a whole. It only makes sense, I suppose, to quantify the success of the game as a product as well.

It turns out, however, that scoring of this sort is a little too complicated for its own good; You might as well task yourself in reviewing a human being, what with all the warts and beauty a game can bring to the table. How would you score your friends, and how would they measure up?

The topic of game review scoring is a hot one. I suppose the basic argument is about what exactly a score means. Is it to determine whether the consumer should make a purchase or not? If so, why not adopt a binary metric such as the thumbs up/down of Ebert & Siskel? Is it to determine how the purchase stacks up to other purchases? At that point you are in the domain of averages, and you end up with lists sorted by score; For a long time The Legend of Zelda: Ocarina of Time was “the best game in the world”, which regardless of how you feel about that game is a patently ludicrous notion to anyone who have any interest in the full spectrum of experiences games can offer.

So the choice appears to be between simpler scoring – “good” or “bad”- and the more elaborate systems, often resulting in scores with decimals. While there are attempts at walking the middle ground between these two approaches, “guide” or “data”, these attempts seem to reduce scoring ranges in the faith that this implies legroom for error and as such should be less contentious. It’s a noble endeavor, but still a compromise rather than a solution to a problem that goes further than the individual scoring mechanic.

Recently, after reading Destructoid’s 10/10 (in actuality 100/100 counting decimals) review of Halo 4, I was struck by how offensive I found that scoring mechanic versus Giant Bomb’s range of 1-5 stars. The implication was, I felt, that even at 5/5 stars the broad strokes of the 5-star range meant there was implied room for flaw, whereas the 100/100 score was too precise to allow any doubt or reason, which are profoundly important to as subjective an art as video games. In a sense, the larger the range, the more I require the full range to be used, lest the values of that range boil away into a skewed average where none of it matters.

I couldn’t tell you when it happened, but at some point, video game scores became practically homogeneous. I’m not opposed to the idea that games themselves have become homogeneous; Look no further than the past decade’s love affair with the Modern Military Shooter, possibly the worst, blandest thing to happen to video games for as long as I have been playing them, though judging by the success of the genre that clearly puts me in the minority.

That games such as XCOM, a moderately simple turn based tactics game (a genre as common as oxygen in the 90s), can appear as rescuing angels of innovation in the year 2012 unfortunately speaks less to the merits of XCOM and more to the creative flatline of an industry where ballooning budgets and economic recession have put the fear of death into nary every publisher in town.

With such enormous budgets, yet so much fear, predicting success is, again, intensely attractive. If X is 100, and Y is like X, Y should be 100 as well, right? Let’s do another one of those Xes.

The answer, it appears, is to guard our investments with aggregate scores. I’m not inherently opposed to score aggregation. As a consumer, I find them highly useful. Rotten Tomatoes is a wonderful thing, probably one of my favorite sites today. It works mostly because film reviews work. While there are scoring systems in place for movies, Rotten Tomatoes does not average scores gathered but rather converts every score into a basic thumbs up or down, or fresh tomatoes vs rotten tomatoes. A gushing review or a middling-to-good review are both fresh; One does not skew the other. In the same sense, a vicious rage fest of a review versus a merely disappointed one count for the same.

The real crux of the problem with statitics and game reviews is publishers’ willingness to base their business off this skewed aggregate Metascore. I wasn’t shocked to hear Obsidian’s developers would not receive a bonus payout if Fallout New Vegas didn’t make 90% on Metacritic, but it didn’t make me any less furious, knowing the very first thing about averages and statistics.

Because averages are painfully sensitive to extreme values (the extremes of a data set is how you gauge the entirety of that data, were you to graph it for instance), so-called outliers will throw off entire ranges. Given 200 scores of 90, a single 10 might drag you down to 89 depending on your rounding. No bonus for you, developers! Why? Because a game reviewer dared have a vigorously divergent opinion.

Rotten Tomatoes have eliminated the outlier problem by normalizing the range into a set of binary values. In one fell swoop they have made a range that is intuitive to the viewer yet insensitive to the personality traits of scoring mechanisms or even reviewers themselves. The resulting percentile score is less a precise metric but rather the answer to a question: Out of how many reviewers, how many thought this movie was any good?

Metacritic instead embraces the whimsical granularity of the games press, adopting Destructoid’s to-me-problematic 100-point range, and as a result, outliers are a cause of great concern. The actual website is fine about it, presenting up front the highest scoring, the lowest scoring, and then someone from the mid range. As a consumer, looking through aggregated reviews, these are the ones I actually care about.

I am much more likely to read “bad” reviews of products simply because they tend to be the more impassioned. It is easier to disagree with a bad review than to disagree with a positive, though that might just be my personality that makes it so. Regardless, I look to outliers to gauge myself on that spectrum. Games are not as easily quantifiable as film; I’ve been burned much too many times on trusting the common consensus (Metal Gear Solid 4 is still the biggest piece of shit still in my collection, take that Metacritic average).

A range is only useful when every value on it has a meaning. Some outlets prize themselves on their willingness to apply the full range, while others take the more politically inoffensive approach of skewing the range towards the positive – everybody knows a game scored 6/10 is pure garbage, right? Combined with the games press’ love affair with granular statistics, this further devalues an average, as nobody seems capable of agreeing on what range they are operating, while quietly refusing to acknowledge their scores are being aggregated and used to drive the industry.

Sigh.

There are numerous further issues with Metacritic, such as their normalization of disparate ranges. For instance, a 1/5 translates to a 20/100, which is in conflict with sites that use the full 100-point range. I shudder to think how Metacritic would interpret a binary system.

Yet none of these issues with Metacritic as a platform would be affecting the industry if it hadn’t been for publisher analysts using the aggregate as a metric for success. Because it is not a metric for success. It is a statistical guesswork based on opinionated guesswork, normalized and processed and skewed by a conflicted press. It barely qualifies as statistics.

And so, Tom Chick’s 1/5 review of Halo 4, actually a good and informative read if a little personal, becomes controversial, with analysts and game developers up in arms about how he dares to write such “look-at-me journalism” (in the words of an enraged David Scott Jaffe) knowing the real-world “value” of the Metascore, or on the flipside, how Metacritic knowing the value of their metric dares include such outliers in their measurement.

For as long as Metacritic’s score average is taken so seriously and given such real-world implications, nobody wins. Not the press, not the developers, and certainly not the consumers.

Edited by Nightriff

MGS4 is one of the greatest games this generation of consoles. Period.

Posted by algertman

@Nightriff said:

MGS4 is one of the greatest games this generation of consoles. Period.

no

Posted by Antikythera

@Nightriff said:

MGS4 is one of the greatest games this generation of consoles. Period.

+1

Posted by CptBedlam

I thought that review was poorly written but I welcome any factors that make metacritic less reliable for publishers. More unpredictable review sources being included on metacritic means it is less likely that publishers will make bonuses and shit dependent on the metascore.

Posted by Donkeycow

@Antikythera said:

@Nightriff said:

MGS4 is one of the greatest games this generation of consoles. Period.

+1

-1.... NO

Posted by SomeJerk

The sooner publishers realize they're digging their own graves by listening to metacritic the better, bring on the one-star reviews.

Posted by mordukai

@Donkeycow said:

@Antikythera said:

@Nightriff said:

MGS4 is one of the greatest games this generation of consoles. Period.

+1

-1.... NO

*Grabs the popcorn and the beach chair*

This should get interesting.

Posted by ImHungry

Thanks for the read. I thought the review itself was fine in its criticisms as it reflected clearly what he found important to Halo and didn't get it, but regardless of how he feels giving it 1/5 is entirely too sensationalist simply as a matter of comparison to other games that get 1/5s

Posted by Bell_End

@SomeJerk said:

The sooner publishers realize they're digging their own graves by listening to metacritic the better, bring on the one-star reviews.

wouldn't it be better just to do away with the arbitrary number at the bottom of a review instead.

Posted by fuzzypumpkin

@Antikythera said:

@Nightriff said:

MGS4 is one of the greatest games this generation of consoles. Period.

+1

+infinity.....I win

Posted by Sunjammer

I think the problem is less about scores and more about how those scores are interpreted and applied. I'm pretty happy with the GB 5-star range, much more so than endlessly elaborate sets of numbers (8.2/10 gameplay!). The real problem is publishers using metacritic as a measure of success and how that shapes the way games are made and how games journalism is conducted.

Posted by Otogi

@Nightriff said:

MGS4 is one of the greatest games this generation of consoles. Period.

I love that this is the first post. Oh, Giant Bomb forums, please change someday.

With that said, though, this was a great read and even though I've heard the argument before, it's fantastic to have a perspective such as yours on it. You've got a follow fro me, and I can't wait to see more posts.

Edited by OldGuy

@Bell_End said:

@SomeJerk said:

The sooner publishers realize they're digging their own graves by listening to metacritic the better, bring on the one-star reviews.

wouldn't it be better just to do away with the arbitrary number at the bottom of a review instead.

Of course it would but then (just as with a recurring topic here on GB made by those too lazy to actually read a review) you'd have an endless stream of complaints that there wasn't a score.

It wouldn't solve the problem anyway, it'd just fall to Metacritic to assign a score based on their best guess (should they choose to include that sites reviews).

Posted by Warchief

@Nightriff said:

MGS4 is one of the greatest games this generation of consoles. Period.

BWAHAHAHAHA just no, it is not.

Posted by WinterSnowblind

@Nightriff said:

MGS4 is one of the greatest games this generation of consoles. Period.

MGS4 was a game?

Posted by Sunjammer

Starting to think I should have edited out the MGS4 line :-P

Posted by Nonapod

@Sunjammer said:

Unfortunately for everyone, consumer and industry, “success” is currently measured in the Metascore

I get the point you're making but this statement isn't exactly true. While a games Metascore is certainly a small factor, strictly speaking from the industry's standpoint a game is successful if it makes tons of money. Reviewers can dump on a game (or movie or music album or TV show or book ect.) all they want, but if it still sells well and makes a hefty profit, then it was a success.

And if game companies are using their own internal reviewers to determine if an early build of a game will score well, I look at that as another level of QC that will most likely tend to lead to better games. After all, if someone is pointing out the glaring problems a game has earlier on, it's more likely that they will be fixed before release.

But I certainly agree that Metacritic is pretty far from an ideal quantifier of a games worth, I'm just not convinced that the game developers themselves are relying too heavily on it at this point.

Edited by cexantus

And it all goes right back to the problem with attaching scores to reviews. I think sites like Giant Bomb should just get rid of the 5 star or 100 point scale and let the review speak for itself. The problem here is of the implication that "X game that gets an average of 80% is just as good as Y game that also received an 80%, which is why so many forum posts are filled with "CALL OF DUTY GETS THE SAME GRADE AS *insert favorite game here*? THAT'S BIAS!!!!!!" I mean, what exactly does a four-star rating or a perfect score even mean? Game criticism, or any criticism for that manner, is such a wildly subjective field that attempting to deduce game reviews to a percentage is absolutely crazy.

@Sunjammer:Yup. Seriously, calling out one of the more popular of this generation is a surefire way to make sure people nerd rage all over this thread.

Edited by Sunjammer

If publishers withhold bonuses if a Metacritic score requirement is not met, I find it hard to believe developers don't pay close attention. Of course it's all anecdotal conjecture, and I'd be happy to be wrong.

well I was trying to use it as an example of a popular game not necessarily sitting right with everyone...

Edited by Bos1014

I like the way rotten tomatoes does the fresh and rotten, im surprised that it hasnt jumped on the gaming bandwagon and just used the same template for games that it does for movies. would be much less rage from fans for games such as Halo 4 would be getting 95% Fresh on RT than an 87 on MC

Also this was a really good read thank you.

Posted by Veektarius

Metacritic is not the problem, and it provides an informative perspective. Yes, the average is skewed by outlying reviews, but any major release has enough reviews to overcome a few 'look at me' journalists. I challenge you to point at an individual metacritic review with a sufficient number of observations (I'd say 40, arbitrarily) and tell me that the score does not reflect the general critical consensus on that game, where green is 'near-universal acclaim', yellow is 'imperfect but enjoyable to many' and red is 'don't bother'.

It may seem that AAA games are rated on a different standard, but when you consider that the production values on these games are often astronomical, it decreases the range of sources from which a decrement could originate. Unlike a movie, which is rated mostly for its story, its artistic vision and the performances of its actors, a game is rated primarily on A) How well it controls, which will improve with testing B) Its technical proficiency, which will be improved by money in the same way as A, and C) How fun it is, which is the most subjective part of the rating, and often (I think as a result) receives the least weight in a reviewer's score in a game where A and B are excellent.

Your example of Fallout: New Vegas is a bit of a strawman. yes, it was an enjoyable game, but it's not as if Obsidian wasn't paid for it. We're talking about a bonus they would have received for delivering a top-tier product. Yes, New Vegas probably came closer to that goal than anything else Obsidian has done, but the critics who knocked it had legitimate reasons for doing so. A threshold has to be put somewhere, whether it's a Metacritic score of 9 or a rottentomatoes score of 85 or a total sales of $5 million, and wherever that threshold is, it's going to be possible to just miss it.

The RottenTomatoes score is definitely not a superior execution to metacritic. What it does is that it deletes information from the critics' individual models before aggregating them. This is necessary in the film world because there are critics who do not subscribe to a numbered system and cannot be combined except through such a simple model. However, in the end, your result is that Inception gets a Rotten Tomatoes score of 86% with an average critical rating of 8 and Looper gets a score of 94% with an average critical rating of 8.1. Your mileage may vary, but my conversations with others support my opinion that a nearly 10 point disparity in favor of Looper does not reflect people's generally mildly warm reception of the movie.

Much like in movies, a positive critical reception is only a means to an end, and that end is making money. Let's consider a two by two set of possible outcomes. A game can receive good or bad critical reception and good or bad public reception. If a movie, or a game, makes enough money, more games like it will get greenlighted, regardless of its critical reception. That allows two of the four possibilities to be a viable path for a game. The consideration of metacritic can only help good ideas by allowing these publishers to notice when a game has untapped potential that did not translate to sales... this time, but which might if attempted again with better marketing. This makes a third of the four conditions viable. So, not only do I say that Metacritic is valid, but I say that it is good.

Posted by Nonapod

@Sunjammer said:

If publishers withhold bonuses if a Metacritic score requirement is not met, I find it hard to believe developers don't pay close attention. Of course it's all anecdotal conjecture, and I'd be happy to be wrong.

I can't speak to what might be going on internally between publishers and developers either, but if they are indeed offering or withholding bonuses based if a game reaches a particular Metacritic score or not, it doesn't seem like a good system of evaluation of a particular developers work. A more logical system would be if a game reaches a particular sales goal, or simply giving developers actual revenue shares.

Posted by Nightriff

@Sunjammer said:

Starting to think I should have edited out the MGS4 line :-P

Probably for the best, I just couldn't resist once I hit that line, sorry for derailing the board

Posted by Kidavenger

@Sunjammer: Good post, I think you put a bit too much blame on Metacritic though, it's a simple tool that has become a bit too powerful in the market so now people with influence are playing with it, manipulating it's flaws. This points to a problem with no reasonable solution other than scraping the whole thing which isn't going to happen, so why go there?

I wish there was some way to hold reviewers accountable for their scores, high or low. I'm no fan of Halo, or console shooters in general, but I think anyone giving Halo 4 the lowest possible score is completely misguiding their audience and everyone else that may be exposed to their "opinion" through Metacritic.

Edited by Sunjammer

I disagree with almost everything you said :-) Metacritic IS okay as a site from a consumer point of view, but publishers applying its scoring as a precise metric is the problem. RT eliminates outliers, that is why i brought it up, whereas MC does not, and halo sees a sub-90 score as a result. As far as I gather, a sub-90 metacritic score denies you a bonus. My post is brought on specifically due to outrage by David Jaffe and analyst Kevin Dent on Twitter, outrage posed from a developer/industry worker point of view.

There is no defending the use of a capricious, imprecise metric as a contractual requirement for a bonus. It is ridiculous for developers to agree to such a clause, unless they fully expect not to be paid that bonus. I bring up New Vegas because Halo 4 and 343i now finds itself potentially in the same boat sub-90 boat due to a fundamental flaw with Metacritic as a mechanism.

Otherwise I find the notion that a critical darling selling poorly would translate to a retread in marketing to hit untapped sales potential for a sequel is naive at best. The road to Modern Warfare 4 is littered with the corpses of critical darlings dead on arrival. It simply doesn't happen that way.

Further, you ignore that MC does worse than delete information from individual models; It infers information. The 1/5 in question counts for 20 metacritic points. That is what Metacritic infers, not what 1/5 means to the reviewer. Inference is a much greater sin than simplification.

(As for Inception being better than anything other than maybe Samurai Cop, and that's a stretch, I think bringing up specific A vs B cases will almost always be moot points)

Edited by Sunjammer

Just so I'm clear; I don't think Metacritic is bad. I think the way its resulting metric has implications for the industry and press is bad, due to the implied precision where there is little precision to be found.

Posted by Ghostiet

@Bell_End said:

@SomeJerk said:

The sooner publishers realize they're digging their own graves by listening to metacritic the better, bring on the one-star reviews.

wouldn't it be better just to do away with the arbitrary number at the bottom of a review instead.

Exactly. If a number of outlets decided for a idiosyncratic rating system - like, I don't know, giving out only a "skip, rent, buy" "score" at the end of the review - the problem would at least be less pronounced. Maybe it would teach people to actually read reviews instead of paying attention just to the number of stars or shit.

Giant Bomb's review system shows what's broken about this shit. It's a 1-5 grade only on paper - in practice, there's a certain nuance, where 3 stars don't mean "mediocre" but "it requires a certain taste to enjoy".

Posted by Bones8677

@Bell_End said:

@SomeJerk said:

The sooner publishers realize they're digging their own graves by listening to metacritic the better, bring on the one-star reviews.

wouldn't it be better just to do away with the arbitrary number at the bottom of a review instead.

That would be throwing the baby out with the bath water. Scores are very useful, and provide people the kind of definition that some reviews miss. I've read lots of reviews that didn't provide an actual score, and after reading said review I often ask, "So is this game good or bad?" Some reviews are poorly written and vague. Scores have a way of fixing that to provide an overall feel. It's like using a ruler as opposed to saying "Well it's about yay long."

Posted by Veektarius

@Sunjammer said:

I disagree with almost everything you said :-) Metacritic IS okay as a site from a consumer point of view, but publishers applying its scoring as a precise metric is the problem. RT eliminates outliers, that is why i brought it up, whereas MC does not, and halo sees a sub-90 score as a result. As far as I gather, a sub-90 metacritic score denies you a bonus. My post is brought on specifically due to outrage by David Jaffe and analyst Kevin Dent on Twitter, outrage posed from a developer/industry worker point of view.

There is no defending the use of a capricious, imprecise metric as a contractual requirement for a bonus. It is ridiculous for developers to agree to such a clause, unless they fully expect not to be paid that bonus. I bring up New Vegas because Halo 4 and 343i now finds itself potentially in the same boat sub-90 boat due to a fundamental flaw with Metacritic as a mechanism.

Otherwise I find the notion that a critical darling selling poorly would translate to a retread in marketing to hit untapped sales potential for a sequel is naive at best. The road to Modern Warfare 4 is littered with the corpses of critical darlings dead on arrival. It simply doesn't happen that way.

Further, you ignore that MC does worse than delete information from individual models; It infers information. The 1/5 in question counts for 20 metacritic points. That is what Metacritic infers, not what 1/5 means to the reviewer. Inference is a much greater sin than simplification.

(As for Inception being better than anything other than maybe Samurai Cop, and that's a stretch, I think bringing up specific A vs B cases will almost always be moot points)

To address your points succinctly

1) The Halo 4 and Fallout: NV cases are actually very similar. They are both games that received some criticism for being more of the same without sufficient improvement on visible flaws. This merits a sub 90 score in my mind, and sends an important message to publishers that greater innovation is necessary.

2) Critical darlings that do poorly may or may not get greenlighted for sequels. It's definitely not a 100% proposition, but it's the reason that companies like Double Fine kept getting work, and the reason that DarkSiders got a sequel.

3) Translating a 1/5 score to 20% is probably legitimate. He gave it that score because he thought it was far, far, from a good game. Whether his opinion should be considered legitimate is another problem. JG has said that he feels the metacritic translation is appropriate for the GiantBomb 5 star system.

4) Criticizing my A v B example calls into question your Fallout: NV vs. MW4 comparison.

Posted by WMWA

I actually really like the idea of adopting Rotten Tomatoes binary system. Never thought of it before

Posted by YOU_DIED

@Sunjammer: the bits about Rotten Tomatoes' system was pretty interesting, how about we start Rotten Potatoes for vid games?

Posted by RecSpec
@Nightriff said:

MGS4 is one of the greatest games this generation of consoles. Period.

Sorry, no. And I love the MGS series.
Posted by Sunjammer

@Veektarius: I think we can agree to disagree on the value of inferred information :)

As for Halo 4's metascore, considering it took an extreme outlier with intensely personal criticism of the game to bring it sub 90, I think it's fair to call that sub-90 average into question if it is presented as a precise gauge of game quality.

And I never compared NV to MW4.

Posted by MordeaniisChaos

Any review of Halo 4 that gives that low of a score is either being disingenuous or is coming from a perspective totally useless as a reviewer. If you are going to give Halo 4 that score, you knew from the very beginning you'd feel that way about that way, and you were never going to give a score based on the game's merit because all ya see is a game you'll never be able to like. It's why big sites like IGN and Gamespot have people who review genre games. You wouldn't have Patrick review Call of Duty or Forza Motorsports games.

I dunno if this is the case with this review, but no matter what, no sensible and fair review of Halo 4 would give it a score so low.

Reviews are an opinion of an individual to a point, but if the game is actually well built, no matter how far from your taste it is, it should never get a very low star.

Posted by BigBoss1911

@Nightriff said:

MGS4 is one of the greatest games this generation of consoles. Period.

I could go one forever talking about how incorrect this is.

Posted by doobie

that review was the kind of review i would expect to see as a user review on gamespot from some bitter PS3 fanboy that just wants to bring down the average score. even if you hate a game/franchise you should still be able to recognize it as a quality piece of software and halo 4 is a quality piece of software.

Posted by CrossTheAtlantic

@Bos1014 said:

I like the way rotten tomatoes does the fresh and rotten, im surprised that it hasnt jumped on the gaming bandwagon and just used the same template for games that it does for movies. would be much less rage from fans for games such as Halo 4 would be getting 95% Fresh on RT than an 87 on MC

Also this was a really good read thank you.

Does it not anymore? I remember a few years ago, it used to. It also mentioned that video game scores were generally inflated, so I think they counted anything below a 70 as rotten.

Posted by Aetheldod

All I wish to say that developers give only to 90%upward metacritic score bonuses because they are stingy ..... no reason for them to give more money to their employes unless they think they are getting more sales .... which in theory a 90% + grade would give ... but we all know that is BS in capital letters.

Also do what I do ... DO NOT GO TO FUCKING METACRITIC

Just see vids of the games/quicklooks/Lets play etc. (now very prominent because of Youtube) and see the game in motion or try out a demo , but we all know demos are scarce and rarely a good representation of a game :/ sometimes

Posted by WickedFather

Three stars with no zero stars makes more sense. And to make it even more sensible every so often there should be a 4th star. But very, very rarely.

Posted by dcgc

@BigBoss1911 said:

@Nightriff said:

MGS4 is one of the greatest games this generation of consoles. Period.

I could go one forever talking about how incorrect this is.

Please, by all means...

Posted by fox01313

I would just love if publishers & studios in the game industry would pay attention to the sites like gamerankings (followed by reading the reviews) where it just gives the score & collects links based on game journalism sites so viewers on the site can read reviews without worrying about the ever present flood of spiteful or fake reviews put in by some of the people on Metacritic.

Posted by geirr

I've never gone to Metacritic before, but today I've gone there twice due to links on these forums. Great.

@dcgc said:

@BigBoss1911 said:

@Nightriff said:

MGS4 is one of the greatest games this generation of consoles. Period.

I could go one forever talking about how incorrect this is.

Please, by all means...

MGS4 came out at a time of wonder and awe of the PS3; if not the best experience it was certainly a crazy experience and one I'll likely not forget, ever.

Posted by TheHBK

@Nightriff said:

MGS4 is one of the greatest games this generation of consoles. Period.

No its not. Game? barely. Movie yes. But great, not even close. Fuck Hideo Kojima and his half assed story telling techniques.

Posted by makari

@Nightriff said:

MGS4 is one of the greatest MGS games this generation of consoles. Period.

Posted by Sauson

I like how this became an argument over MGS4.

Good job GB

Edited by CookieMonster

Does metacritic literally just take the average score? I'm sure there must be more to it than that. Otherwise, its more of a fucking dumb website than I thought it was. Also, good read.

Posted by Tackchevy

I approve of this message. Thumbs or stars work best for gauging relative excellence in the context of the times. Averages suck, especially averages that impact pay for hard working people.

Posted by HoboZero

Man, I do love a good statistical analysis :) Doesn't the binary system used by Rotten Tomatoes raise the question of exactly what constitutes a "Good" or "Bad" score? I'm not sure what their methodology is - does the reviewer or publisher have to specify that a review is good/bad, or is it determined by the score; how do you translate 2 out of 4 stars, or one thumb up and one thumb down? I'd wonder what that would do to a 5 star scale like Giantbomb's; I don't necessarily think of 3-stars as "bad". But, as you say, the likelihood of Metacritic changing their methodology seems remote. Maybe RottenTomatoes should branch out into game meta-rankings?

Publications need, or think they need, the hits from aggregators like Metacritic, so they'll continue to participate And I don't think reviewers "poisoning the well" with more score outliers is necessarily the best way to bring change to the system. The only realistic alternative I see is another party coming in with a different aggregation methodology and supplanting Metacritic, but is that likely?

At any rate, really great read (and agreed on Metal Gear 4 being kind of shit, although I am still very glad to have played it, because that was some unique, crazy-ass shit :)!

Posted by GERALTITUDE

Good write up! I had to laugh when, after reading the whole thing, the first comment was just MGS4 is sweet or whatever.

Which it fucking is, but yeah, ok, I can deal with you being the outlier on that one. :P

I've spent some time thinking about this topic, and certainly couldn't do it anymore justice than you did here. Just to respond to some of the comments in here, publishers withholding bonuses and developers paying attention to Metacritic is not anecdotal or theoretical in the slightest. It doesn't hold true for all pubs and devs, but it does for many of them.

It's hard to say exactly where one problem ends and one begins but Metacritic does matter, which, you know, it shouldn't. It's easy to say this from the outside but once you walk into the system it's another story altogether.

We just heard on the Bombcast for the nth time that Marketers and Public Relations peeps are often on the line for a game's score. This isn't a shit idea drawn out of nowhere, it's drawn from this question: How do I rate my employees? How do I measure their efficacy? How do I justify their salaries? How do I fire a poor employee? How do I justify increasing a salary? Metacritic is one tool to answer these questions, and it'll be important until the whole framework that surrounds it goes belly up, which it won't.

Hate to be Joe I Hate Corporations but until the men signing the checks care about more than the bottom dollar they will always depend on calculators to justify or deny a bonus, raise, project, salary, etc.

  • 61 results
  • 1
  • 2