On Games, Reviews, And Criticism -- Part 2

What it means to be a critic, one's approach to being critical, and how that relates to the larger idea of "criticism" versus a traditional "review" are topics that any writer or developer will give you endless opinions on.

Mass Effect 3 senior designer Manveer Heir and I have been exploring this idea, a conversation prompted by the vocal response to Simon Parkin's Uncharted 3: Drake's Deception review, which served as a review of Uncharted 3 and the entire Uncharted series.

The response was overwhelmingly positive last week, and that makes me awfully happy. You'll see more of this--promise.

In part two, we contemplate whether dropping review scores, which some publications have tried, could be a potential solution. Hmm.

Without further delay, here's the second part of our conversation. If you missed part one, read it here.

Note: This exchange took place over email, and I've done minimal editing to reflect the casual style.

--

Patrick,

The middle ground is simple: we drop review scores. Sorry, I learned my negotiation tactics from the Republicans. But in all seriousness, I think when it comes to reviews you may be right that publishers put a big emphasis on the reviews on their own accord, but publications are the ones that created the overinflated scoring that plagues the industry. My suggestion to fix that is simple. All review scores go to five stars with no half-star given. Make sites make a call on how good/bad a game is and put their name on it. A five star system will not lead to inflation. Anything more than five review points will naturally lead to inflation. You see it on sites that rate out of 10. You definitely see it on sites that rate out of 100. Of course, your industry won't ever change to this in the same way mine won't stop using Metacritic as the metric for all things.

Also, saying that publishers put an emphasis on review scores, while correct, doesn't address the fact that sites still have an obligation to make sure their information is being used in a responsible manner when possible. This means, if a site disagrees with its reviews being used on Metacritic, it should get them pulled from the site or make changes to how its scores are interpreted. I'm not sure how possible that is, but I have to imagine a site can ask to be taken off Metacritic, right? In college football, which I am a big fan of, the AP Poll asked to be taken out of the convoluted BCS formula that attempts to match up the two best teams at the end of the year in a national championship game. They did this because they disagreed with how their poll was being used, and because they understood that by letting the BCS use their poll, it made them, in fact, complicit with the problems that the BCS represented in it's fallacious attempt to match up #1 and #2. This is the part where I would quote Peter Parker, but that's become very hackneyed lately, so I'll just say I think responsibility is on both sides here.

But this whole path would be painful for me as a developer. If, all of a sudden, all reviews were out of five and three was average and everyone adhered to this, a 60% score would be average, instead of 80% as it currently is. That would crush developers/publishers, and customers wouldn't know how to parse the new scores. We've crafted a bed of nails, and now we are lying on it. To try to get up from this will be incredibly painful and difficult.

I think trying to change the system we currently have is incredibly difficult. There is inertia working against us. People will have a hard time adjusting, from the writers, to the developers, to the publishers, to the readers/players. This just creates chaos and will take long to occur, and, as you suggest, not happen anytime soon, if ever. So what's the real middle ground? I think it's to create something wholly new, different. Something that doesn't have the preconceived notions and expectations that reviews have. And I think that thing is called criticism. Yes, I'm repeating my earlier point, but it's because we've pointed out why the "good fight" here is so futile. Our only options are to do something different or just be ok with the status quo. I, for one, am absolutely not ok with the status quo. Not as a reader, and not as a developer.

If we introduce critical analysis, we can train our readers on something new. We'll still have these review things that they want and read, but we'll slowly start putting more emphasis on the criticism. The discussion of authored narrative vs. player-driven. The discussion of the role of modern military shooters in our world. The rise of social networking and connectivity among our games. These are interesting topics, and there are multiple sides to the discussion and these topics can be framed by using specific games to make a persuasive argument. I want to read that. I bet many writers want to write that. And if we offer it to our consumers, I think there will be an appetite for it. But there will not be an appetite for it if we don't offer it. Readers, like players, don't know what they want. They just want what they currently like; they have little vision for the future. If you asked a player after Call of Duty 3 what they wanted, they would have wanted Call of Duty 4 to be more World War II awesomeness. It took someone with vision to take that game to the modern day. And you know what? It seems crazy to think this now, but there was resistance against that and it took guts to make that move. We need an equivalent move amongst the journalists.

So let's not get rid of reviews. But let's offer a side dish of real pointed criticism. And over time, let's make that critical side dish bigger and bigger and more prominent. And, you never know, one day maybe it'll become bigger than the reviews itself. That's the middle ground to me. Something new. Or maybe you have a different idea for something totally new we could do to fill this gap?

--

Manveer,

Why don't you chill out, Gingrich? Review scores aren't going anywhere, just like Metacritic--or an equivalent aggregator--isn't disappearing, either. I'm with you on the five-star scale, though, which is what we have here at Giant Bomb. You can opt out of being listed on Metacritic, which is what Adam Sessler did rather publicly for both G4tv.com and X-Play, after criticizing the organization at the Game Developers Conference a few years ago. Adam is actually a bit like Hulk when he gets angry.

And you're right that a publication should opt-out of Metactitic if they're being misrepresented, but it's no secret that publishers are less likely to provide a publication with review code for a game ahead of release if they are not on Metacritic..

There are few things more frustrating than flipping through a publication with a review scale that goes from 0 to 100. Ugh, ugh. I've never regularly written for a publication that asked me to review a game on that scale, and I can't imagine it, either. If someone can tell me what the difference is between a 72 and a 73, I'd love to know. It makes the reviewers job more difficult, and does nothing to help the developer, an entity whose role is often forgotten when it comes to the job of a review.

I don't believe it's the job of a reviewer to take things like "this may put the developer out of business" when considering a review, but a good review should also serve as a guide for a developer to understand what did and didn't work. A review has failed when the writer becomes a backseat developer, using their soapbox to wax on and off about their design "insight."

And while the battle for "hits" is perhaps another conversation altogether, it's worth noting. Reviews drive an enormous amount of traffic to gaming web sites, and part of that traffic is driven by the score, especially if it's at one end of the scale or the other. That's not to suggest publications are skewing scores in order to generate traffic--the amount of conspiracy theories about the gaming media with any evidence is laughable--but there's financial motive to keep reviews a center piece.

The impasse we're finding ourselves at, I believe, is whether a "review" can also function as "criticism." Games For Windows Magazine (aka Computer Gaming World) dropped review scores for a short period, hoping to force readers to spend more time contemplating the text. They ended up ditching this plan, as people stopped reading the reviews as much. In an experiment under John Davison's leadership, GamePro stopped doing print reviews, and instead aggregated the more timely online reviews to provide an overall perspective of the response to a particular game. That was also eventually dropped.

I'm not advocating that we shouldn't challenge what the reader wants, but that there are expectations. As a developer, you do the same, awkward dance with players. As a reporter, I've constantly pushed against the notion that we should always give the reader what they think they want. If we applied the same philosophy to food, everyone should be a-okay with McDonalds and shut up, simply because it's popular. You demand more when you know what to demand, and it's up to us to provide.

Let me bring around the question to that prompted this dialogue. Are you so opposed to "criticism" being part of a "review" because it's implied that criticism will, inherently, skew more negative? Maybe it's a problem with the label itself. Review come from a weird place where games used to (and, in some cases, still are) broken down by graphics, sounds, gameplay, which is completely unfair, crazy, and should stop. Criticism, however, sounds negative. In the case of Parkin's Uncharted 3 review, he ended up giving the game an 8/10, but as most of his text was spent critiquing Naughty Dog's philosophical approach, he pissed off two sects of people: those who wanted a higher score, those wondering why it wasn't way lower.

It's clear that reviews are in a transitional state, and maybe that's the real problem--it just needs more time?

Look for the final part of our conversation tomorrow, which means I have to work on my day off. Crap.

What it means to be a critic, one's approach to being critical, and how that relates to the larger idea of "criticism" versus a traditional "review" are topics that any writer or developer will give you endless opinions on.

Mass Effect 3 senior designer Manveer Heir and I have been exploring this idea, a conversation prompted by the vocal response to Simon Parkin's Uncharted 3: Drake's Deception review, which served as a review of Uncharted 3 and the entire Uncharted series.

The response was overwhelmingly positive last week, and that makes me awfully happy. You'll see more of this--promise.

In part two, we contemplate whether dropping review scores, which some publications have tried, could be a potential solution. Hmm.

Without further delay, here's the second part of our conversation. If you missed part one, read it here.

Note: This exchange took place over email, and I've done minimal editing to reflect the casual style.

--

Patrick,

The middle ground is simple: we drop review scores. Sorry, I learned my negotiation tactics from the Republicans. But in all seriousness, I think when it comes to reviews you may be right that publishers put a big emphasis on the reviews on their own accord, but publications are the ones that created the overinflated scoring that plagues the industry. My suggestion to fix that is simple. All review scores go to five stars with no half-star given. Make sites make a call on how good/bad a game is and put their name on it. A five star system will not lead to inflation. Anything more than five review points will naturally lead to inflation. You see it on sites that rate out of 10. You definitely see it on sites that rate out of 100. Of course, your industry won't ever change to this in the same way mine won't stop using Metacritic as the metric for all things.

Also, saying that publishers put an emphasis on review scores, while correct, doesn't address the fact that sites still have an obligation to make sure their information is being used in a responsible manner when possible. This means, if a site disagrees with its reviews being used on Metacritic, it should get them pulled from the site or make changes to how its scores are interpreted. I'm not sure how possible that is, but I have to imagine a site can ask to be taken off Metacritic, right? In college football, which I am a big fan of, the AP Poll asked to be taken out of the convoluted BCS formula that attempts to match up the two best teams at the end of the year in a national championship game. They did this because they disagreed with how their poll was being used, and because they understood that by letting the BCS use their poll, it made them, in fact, complicit with the problems that the BCS represented in it's fallacious attempt to match up #1 and #2. This is the part where I would quote Peter Parker, but that's become very hackneyed lately, so I'll just say I think responsibility is on both sides here.

But this whole path would be painful for me as a developer. If, all of a sudden, all reviews were out of five and three was average and everyone adhered to this, a 60% score would be average, instead of 80% as it currently is. That would crush developers/publishers, and customers wouldn't know how to parse the new scores. We've crafted a bed of nails, and now we are lying on it. To try to get up from this will be incredibly painful and difficult.

I think trying to change the system we currently have is incredibly difficult. There is inertia working against us. People will have a hard time adjusting, from the writers, to the developers, to the publishers, to the readers/players. This just creates chaos and will take long to occur, and, as you suggest, not happen anytime soon, if ever. So what's the real middle ground? I think it's to create something wholly new, different. Something that doesn't have the preconceived notions and expectations that reviews have. And I think that thing is called criticism. Yes, I'm repeating my earlier point, but it's because we've pointed out why the "good fight" here is so futile. Our only options are to do something different or just be ok with the status quo. I, for one, am absolutely not ok with the status quo. Not as a reader, and not as a developer.

If we introduce critical analysis, we can train our readers on something new. We'll still have these review things that they want and read, but we'll slowly start putting more emphasis on the criticism. The discussion of authored narrative vs. player-driven. The discussion of the role of modern military shooters in our world. The rise of social networking and connectivity among our games. These are interesting topics, and there are multiple sides to the discussion and these topics can be framed by using specific games to make a persuasive argument. I want to read that. I bet many writers want to write that. And if we offer it to our consumers, I think there will be an appetite for it. But there will not be an appetite for it if we don't offer it. Readers, like players, don't know what they want. They just want what they currently like; they have little vision for the future. If you asked a player after Call of Duty 3 what they wanted, they would have wanted Call of Duty 4 to be more World War II awesomeness. It took someone with vision to take that game to the modern day. And you know what? It seems crazy to think this now, but there was resistance against that and it took guts to make that move. We need an equivalent move amongst the journalists.

So let's not get rid of reviews. But let's offer a side dish of real pointed criticism. And over time, let's make that critical side dish bigger and bigger and more prominent. And, you never know, one day maybe it'll become bigger than the reviews itself. That's the middle ground to me. Something new. Or maybe you have a different idea for something totally new we could do to fill this gap?

--

Manveer,

Why don't you chill out, Gingrich? Review scores aren't going anywhere, just like Metacritic--or an equivalent aggregator--isn't disappearing, either. I'm with you on the five-star scale, though, which is what we have here at Giant Bomb. You can opt out of being listed on Metacritic, which is what Adam Sessler did rather publicly for both G4tv.com and X-Play, after criticizing the organization at the Game Developers Conference a few years ago. Adam is actually a bit like Hulk when he gets angry.

And you're right that a publication should opt-out of Metactitic if they're being misrepresented, but it's no secret that publishers are less likely to provide a publication with review code for a game ahead of release if they are not on Metacritic..

There are few things more frustrating than flipping through a publication with a review scale that goes from 0 to 100. Ugh, ugh. I've never regularly written for a publication that asked me to review a game on that scale, and I can't imagine it, either. If someone can tell me what the difference is between a 72 and a 73, I'd love to know. It makes the reviewers job more difficult, and does nothing to help the developer, an entity whose role is often forgotten when it comes to the job of a review.

I don't believe it's the job of a reviewer to take things like "this may put the developer out of business" when considering a review, but a good review should also serve as a guide for a developer to understand what did and didn't work. A review has failed when the writer becomes a backseat developer, using their soapbox to wax on and off about their design "insight."

And while the battle for "hits" is perhaps another conversation altogether, it's worth noting. Reviews drive an enormous amount of traffic to gaming web sites, and part of that traffic is driven by the score, especially if it's at one end of the scale or the other. That's not to suggest publications are skewing scores in order to generate traffic--the amount of conspiracy theories about the gaming media with any evidence is laughable--but there's financial motive to keep reviews a center piece.

The impasse we're finding ourselves at, I believe, is whether a "review" can also function as "criticism." Games For Windows Magazine (aka Computer Gaming World) dropped review scores for a short period, hoping to force readers to spend more time contemplating the text. They ended up ditching this plan, as people stopped reading the reviews as much. In an experiment under John Davison's leadership, GamePro stopped doing print reviews, and instead aggregated the more timely online reviews to provide an overall perspective of the response to a particular game. That was also eventually dropped.

I'm not advocating that we shouldn't challenge what the reader wants, but that there are expectations. As a developer, you do the same, awkward dance with players. As a reporter, I've constantly pushed against the notion that we should always give the reader what they think they want. If we applied the same philosophy to food, everyone should be a-okay with McDonalds and shut up, simply because it's popular. You demand more when you know what to demand, and it's up to us to provide.

Let me bring around the question to that prompted this dialogue. Are you so opposed to "criticism" being part of a "review" because it's implied that criticism will, inherently, skew more negative? Maybe it's a problem with the label itself. Review come from a weird place where games used to (and, in some cases, still are) broken down by graphics, sounds, gameplay, which is completely unfair, crazy, and should stop. Criticism, however, sounds negative. In the case of Parkin's Uncharted 3 review, he ended up giving the game an 8/10, but as most of his text was spent critiquing Naughty Dog's philosophical approach, he pissed off two sects of people: those who wanted a higher score, those wondering why it wasn't way lower.

It's clear that reviews are in a transitional state, and maybe that's the real problem--it just needs more time?

Look for the final part of our conversation tomorrow, which means I have to work on my day off. Crap.

Good work as always Patrick!

@Daius said:

Good work as always Patrick!

There is no way you've read that already. Wanted to get the quest without being forced to put "FIRST!" eh?

@Chris86: Hey the mere fact he's provided this interview series is good work enough, and I'd doubt that the quality of discourse dropped after the first part.

Also, yes.

Great, part 2 is up ^^

@Daius said:

@Chris86: Hey the mere fact he's provided this interview series is good work enough, and I'd doubt that the quality of discourse dropped after the first part.

Also, yes.

I'm just annoyed because I thought I was going to get the first post for sure. Damn those Giant Bomb guys for creating a quest that turns me into everything I hate about online comments sections.

(Also, great article Patrick, like everyone else I love the angle and style that you've brought to Giant Bomb.)

By now I have ignore reviews since my taste in game grow to niche titles that are fun despite getting low scores. I usually use reviews now to get me a better judgment on a few games that I wasn't sure on. So yeah, let reviews stay as they are helpful.

I'm really liking the dialogue approach, Patrick. A fascinating discussion.

This is the kind of thing I really like about the GiantBomb E3 podcasts, talking with industry folks about some of the behind-the-scenes that goes on when developing or marketing games. You're just doing more of that in a text-based format. Keep it up!!

As a designer I agree entirely with Manveer. Criticism should become a major focus of more sites. There need to be discussions about the nature of game design and game making, but a review is not a place for those discussions. Having those two separate outlets will lead to more innovation on the design front and more rewarding writing for readers and journalists.

More great topics of debate. What's even better is there isn't just agreement on everything either, getting the two perspectives actually makes it a lot more interesting.

Surely gaming is still in such a state of relative infancy as a medium, that not picking up on experience-changing technical differences or detractors would cause fault with games analysis as it is. Games can be played from so many perspectives, and there are so many approaches to what makes a video game a video game, that other broad topics like linearity can't always be talked about without any other bias.

Go Kleptok. I particularly like this paragraph:

"I'm not advocating that we shouldn't challenge what the reader wants, but that there are expectations. As a developer, you do the same, awkward dance with players. As a reporter, I've constantly pushed against the notion that we should always give the reader what they think they want. If we applied the same philosophy to food, everyone should be a-okay with McDonalds and shut up, simply because it's popular. You demand more when you know what to demand, and it's up to us to provide."

There's some deep answers to awkward questions in there. And, yes, reader's expectations do need to be considered especially when reviews can draw in a wider group of people who don't follow a website so closely and may well not be interested in criticism They may just want the score to give them an indication of if they should buy a game they had been interested in and wanted a confirmation on that it didn't suck.

Another thing to consider about review scores is that if you are considered important enough the aggregation sites will assign a score to your review for you (Rotten Tomatoes did -- and, I believe, still does -- this for years for reviews for movies from major papers that did not implement a scoring system)... So, even if all game review sites and magazines dropped scores, someone somewhere would assign them anyway.

I remember when Computer Gaming World dropped scores. I didn't like it. They also put scores from other gaming review publications on the review page. It just came off as pretentious. We're better than some stupid score you stupid reader, so learn to read or stay dumb. Then they came back with letter grade scores. That magazine was all over the place during that time. 
 
I like Metacritic, but I like it because I have a list of reviews from many different sites to choose from and read. It's too bad Metacritic can be used for all the wrong reasons. I'm not sure who's to blame. I guess someone needs to change the culture of how review scores are used.

Can a publication request to be removed from Metacritic?

These articles are great Patrick. Keep 'em coming.

It's really hard to make a decisive call on this issue--without a doubt metacritic and inflated review scores have negatively affected the industry, with developers almost trying to shoot for higher scores just by copying each other rather than attempting to branch out. On the other hand, having all those reviews prevents consumers from spending $60 on a terrible game. So there most definitely is a place for review scores, rather than just text reviews.

I like Manveer's idea of everyone switching to the same scale, however he's right in saying that have an average of 60% would cripple consumer perceptions of games. I personally think a 10 point scale isn't bad, as long as you can't do 7.5 sort of bullshit. People can distinguish between a 6 and a 7, but not a 6.8 and a 7.2.

I'd like to see Giantbomb try to take some ideas from this into their reviews. Maybe they should try hiding the number of stars the game gets underneath a spoiler warning, or at least put them at the bottom of the article in an attempt to encourage people to read the actual text.

I heart the Sees. Anyways scores aren't going anywhere nor should they. It is the readers responsibility to read the review and understand the score and you know what, if they choose not to because the review is 9 pages long and they get pop ups every page I totally understand that. Forceing readers to consume content a certain way is never a good idea. If reviewers wrote better Abe didn't get caught up in their word counts or click throughs maybe people would read more.

Meant the Sess but the phone "corrected" me and I cant edit on the mobile site.

Brilliant article again Patrick, looking forward to seeing more of this but like I said on the first part I can't help but fell that Manveer is possibly preparing us for Mass Effect 3 not being received by the press as well as expected.

I'd love there to be more critical assessments of games in our industry.

Fantastic Patrick. Absolutely loving these articles and this is what I love about Giant Bomb. Getting into the behind the scenes stuff with the Developers and expressing their true opinions on the industry.

Keep up the Great Work!

One of the issues I have with what Manveer was saying is that the 3-star average means 60% is the mean average for all games. That thinking is still locked into the whole percentage method of reviewing. Stars don't need to be a percent of 100. If you want to move away from approximate scores, don't apply a numerical value to a star scale.

Review scores are good for a short hand summation of what the content of the review was trying to convey. They don't really mean much else outside the context of the site when you really think about. The Giant Bomb reviews are fine and dandy but if I really want to know about the game on the site I listen to the Bombcast. Most of the games they talk about are never reviewed in the traditional sense yet I hear more about a game's pluses and minuses during a discussion with several people who have played the game. We need more of that in the industry.

Patrick is right in one big aspect; the reason sites like Giantbomb and Xplay use the 5 star scale is because it does not inflate. You have Horrible (0 stars), Bad (1 star), almost average (2 stars), just above average (3 stars), great (4 stars), and AMAZING (5 stars). And unlike what the 0-10 and 0-100 scales are doing right now, the average is close to the median (the middle score. This has always bugged me, how the average score of games could be at 77%. Why not use the whole scale? Why not give an average game (Singularity) a 50%? Why not give a game that did some good things but still was not up to par (Homefront) a 30%? I look at the other review aggregates on Metacritic like movies and tv shows, and see the average score being below a 50%... but still close.

We need to get journalists and reviewers together and make a stand. The scores need to drop. Until then, reviews seem arbitrary. In fact, metacritic could do something cool like average all of the scores from a site, and then shift those scores to be closer to the 50% average.

Btw, someone should point out the obvious fact that in a 5-star system like GB uses there's actually only 5 different scores. Translating that to 20, 40, 60, 80 and 100 doesn't make sense. It should be traslanted into 0, 25, 50, 75, 100. By making the lowest possible score higher than 0 the translation is already pretty invalid.
 
That's not to say I support the whole forcing 5 star system into a 100 point scale at all.

Am I the only one that thinks Manveer is talking out of his ass with the review vs. criticism thing? Surely you can't have one without the other...right? The review is there to point out the good things and criticise the bad things. If the reviewer finds the design of a game to be boring, isn't that as valid a criticism as saying the graphics suck balls?

What would a criticism free review even sound like? Every single such review would boil down to something like "when you push the button the guy moves, 5 stars".

Shouldn't Giant Bomb get taken off Metcritic then? It always bugs me when the 3* review gets put up as 60 and if Patrick you are so adamant about this being wrong why not get Giant Bomb removed - or are you worried about losing hits from linkthroughs?

As regards the overall article I don't think there is as much difference as you make out between reviews and criticism. In a mature artform they are close to being the same thing. I think the reason the Eurogamer review was so contentious is that it was disparaging Uncharted for being too linear when that was something it was never supposed to be. It is the same problem that plaques broadsheet reviews of genre and action movies. If something only intends to be a fun bit of popcorn is it acceptable to damn it for lacking the ambition to be something deeper.

I think this argument comes round in the end to know your reviewer and accepting that their opinion is just that. The problem is that when this opinion is sealed with a score out of 100 broken down into categories it gives the impression of being way more scientific than it actually is.

@Raven10: I have to say that I don't agree with you or Manveer on this. I think criticism is a vital part of reviewing a game, and that in a perfect world, the solution would be for Metacritic to stop being used the way it is by publishers. I don't think the problem lies with how games are reviewed, or with what writers take issue with in a game, but rather with people trying to take advantage of a system that was never meant to be viewed as a congregate entity. Saying criticism doesn't belong in a game review is like saying criticism doesn't belong in a movie review. I just don't see it.

@leebmx said:

Shouldn't Giant Bomb get taken off Metcritic then? It always bugs me when the 3* review gets put up as 60 and if Patrick you are so adamant about this being wrong why not get Giant Bomb removed - or are you worried about losing hits from linkthroughs?

I haven't read the article as I am saving it for later, but Jeff has said on formspring that he thinks metacritic works with GB's scoring system.

Header: "Patrick and Mass Effect 3 senior designer Manveer Heir contemplate the idea of dropping review scores,"

Patrick, in the first line of his response: "Review scores aren't going anywhere, just like Metacritic--or an equivalent aggregator--isn't disappearing, either."

Pffft. Well, at least one of you contemplated it.

If game developers are unhappy by the way their game is being reviewed, they should at least release a demo for it.

I do turn to game reviews to decide whether I want to buy a game or not, but it always takes a backseat if the game has a demo. If developers take the time to release a proper demo for their game they would not be at the mercy of game reviewers and Metacritic.

If a game does not have four reviews by the reviewers recognized by Metacritic, a game will go unnoticed there. If the same game has a demo, the chance of it being spotted and judged by the players themselves increases greatly.

@SuperSambo said:

@leebmx said:

Shouldn't Giant Bomb get taken off Metcritic then? It always bugs me when the 3* review gets put up as 60 and if Patrick you are so adamant about this being wrong why not get Giant Bomb removed - or are you worried about losing hits from linkthroughs?

I haven't read the article as I am saving it for later, but Jeff has said on formspring that he thinks metacritic works with GB's scoring system.

That's just the thing... Metacritic works better with Giantbomb's and Xplay's 5 star system. A 3 star game is the equivalent to a 60/100. The problem is not these scores, but all of the others that use higher scales but average the scores to 78%. These are the scores that need to change.

No Steam, No Sale, Bioware.

I approve this discussion. I would like more.

@sanchopanza said:

Am I the only one that thinks Manveer is talking out of his ass with the review vs. criticism thing? Surely you can't have one without the other...right? The review is there to point out the good things and criticise the bad things. If the reviewer finds the design of a game to be boring, isn't that as valid a criticism as saying the graphics suck balls?

What would a criticism free review even sound like? Every single such review would boil down to something like "when you push the button the guy moves, 5 stars".

To me (and maybe I'm in the minority on this), reviews are more like quick looks in text form. They express facts and features or a product, and in the end, rate it. In a review, the author has to stay objective. With a critique, the author's background is taken into account, and he/she is free to express their complete opinion about a game. Podcasts act as great mediums for critiques, but as journalists are becoming more known, people understand their tastes, and can come to reasonable conclusions based that person's biases.

@SuperSambo: @Korne said:

@SuperSambo said:

@leebmx said:

Shouldn't Giant Bomb get taken off Metcritic then? It always bugs me when the 3* review gets put up as 60 and if Patrick you are so adamant about this being wrong why not get Giant Bomb removed - or are you worried about losing hits from linkthroughs?

I haven't read the article as I am saving it for later, but Jeff has said on formspring that he thinks metacritic works with GB's scoring system.

That's just the thing... Metacritic works better with Giantbomb's and Xplay's 5 star system. A 3 star game is the equivalent to a 60/100. The problem is not these scores, but all of the others that use higher scales but average the scores to 78%. These are the scores that need to change.

Interesting....I don't think it works at all. After all we surely can't be saying that a 5/5 on GB is 100% game. I bet GB has a way higher number of 100 games on metacritic and that can't be what Jeff etc really want the world to think they are saying in their reviews.

Maybe they don't care, which would be fine as Metacritic would be silly if it wasn't for all the power that has been invested in it, but I can't see how they can think the 5 point scale makes sense extrapolated out to 100.

EDIT: don't misunderstand me I think the 5 point scale is the best if you have to use one - which I would prefer not in truth - however it just gets courrpted by Metacritic turning it to a 100 point scale.

My respect for giantbomb would rise to infinite ammounts if they would decide to drop review scores. But I know, no matter how "different" they think/say they are that will never happen. Too bad.

@msavo said:

Can a publication request to be removed from Metacritic?

Absolutely. Just a much as a publisher can call into question user reviews without a worded explanation that crop up as in the case of Bastion.

Articles like this are why I come to Giant Bomb, fantastic work Patrick. Recently I've come to have many problems with video game reviews, as anyone should if they have visited System Wars...It seems to me that reviews have become more about financial business than doing what they are supposed to be which a critique of a work of art. Not going into depth about what makes a game good or bad is not helpful to developer. Take for example a negative review of Star Wars Episode I: The Phantom Menace that was in a newspaper, could one real detail why it was a bad film or what made it bad? Probably not, now if you watch RedLetterMedia's review of The Phantom Menace, even if you completely disagree you are able to detail exactly why the film failed as not only a Star Wars film but as a film as a whole and gives great incite to film makers, Star Wars fans, film enthusiasts and generally everyone interested. I'm in the camp that review scores should just be dropped, which I believe would put more emphasis on the content of the review.

Again going back to my Phantom Menace example, if RedLetterMedia would have attached a numerical rating to his review it would have distracted from the content of his review and instead put the focus on the "bottom line", the number. If someone rights a detailed criticism and critique of game and then states "This game deserves a seven out of ten", well then people start to ignore what was in the review, ignore what may or may not be a problem to them, and instead focus on the number and call it terrible game because it did not score higher. When Tom McShea reviewed Skyword Sword, how many people do you think actually read his review and read his criticisms of the game? Probably not many, they just saw the 7.5 score and then ran to the nearest forum to describe how the game is a horrible flop and how outdated the game series is and how horrible the controls are, without actually have played the game or read what makes the controls "cumbersome" to Tom McShea. If you were a Zelda fanboy though, you may have cried that the game deserved a higher score and that Tom McShea has no credibility and that the controls are fine without reading his review and seeing the faults of Skyward Sword or the controls. I simply think that with scored reviews, scores become to much a focal point, rather than the actual content of the review.

Off the topic of review scores, Adam Sessler is awesome, probably one of the few good things about G4TV and X-Play.

@leebmx said:

@SuperSambo: @Korne said:

That's just the thing... Metacritic works better with Giantbomb's and Xplay's 5 star system. A 3 star game is the equivalent to a 60/100. The problem is not these scores, but all of the others that use higher scales but average the scores to 78%. These are the scores that need to change.

Interesting....I don't think it works at all. After all we surely can't be saying that a 5/5 on GB is 100% game. I bet GB has a way higher number of 100 games on metacritic and that can't be what Jeff etc really want the world to think they are saying in their reviews.

Maybe they don't care, which would be fine as Metacritic would be silly if it wasn't for all the power that has been invested in it, but I can't see how they can think the 5 point scale makes sense extrapolated out to 100.

EDIT: don't misunderstand me I think the 5 point scale is the best if you have to use one - which I would prefer not in truth - however it just gets courrpted by Metacritic turning it to a 100 point scale.

This is where it gets weird. 100% does not mean a perfect game by no means, but yet it is used so frequently in gaming. And you'd be surprised how many sites give waaaaay more 100% than giantbomb. But really, if you want to simplify the scale (aka no more 93s), you have to allow the entire scale to be used. This includes 100% and 0%. I think that's all I want. I'm tired of the 60-100 scale... I want to see the full 0-100... and I want 50 to be the score of an average game. If the average of all scores given is in the upper 70s, something is wrong.

Your shitty review cost my company jobs!!

Your shitty product cost my readers money!!

Moral of the story: Don't make shitty games.

@sanchopanza: I think you're misunderstanding what criticism is being talked about here. Of course reviews will have criticism in them. As I understand it they're referring to the criticisms of outside decisions on a game or the industry in general effecting the game review. I.E. the example they keep referring to with Uncharted 3 where the criticism is about Naughty Dog's decision to continue on with the same formula and keep doing more of the same. While it works and Uncharted 3 is a great game, the criticism is that it's not different even if it's still fun and good. The criticism is about the industry as a whole and not about a flaw of the game specifically. Which is represented in the 8/10 score it got even with that in the review. Manveer is suggesting that type of criticism should be its own thing and not part of a review. Whereas criticisms such as bad controls or bad voice acting or something specific to the game should absolutely be in the review.

I'm loving the article and the discussion it creates however I feel like in this particular one Klepek didn't even acknowledge Heir's point about criticism of the larger things being a separate thing from reviews and continued on arguing why criticism should be in reviews. Not bashing Patrick, love the guy's stuff. I just feel like he kinda missed the whole point of what Heir was trying to say in this e-mail back and forth.

Is part 2 going to be the final part? I figured Manveer would get the last word, or at least we'd see a more formal end to the conversation.

@sanchopanza: No, you are not alone in that. In the first exchange, Manveer had a misunderstanding of what criticism means when applying it to a medium. It does not mean to point out only the negatives (which is what he believes), but to critique (as Patrick believes and practices, given his Skyward Sword review). But now Manveer is advocating a two-tiered review system that serve anyone. It doesn't serve the publications, as reviews without scores go largely unread. And it doesn't serve the reader, as the system segregates critical thinking and reductive reasoning into two different pieces, the latter of which would be more widely read.

I think he's backed himself into a corner by stating earlier that criticism has no place in a review (a ridiculous argument), and now he's making up absurd alternatives when the logical end to his argument is the realization that criticism has a place - a vital place - in a review. The simple solution is thus: write better reviews. Unfortunately, that simple solution is difficult to achieve. A well-written review receives the same amount of hits as a poorly-written review, so what's the incentive (from the publication's point of view) to seek out intelligent writers?

@sanchopanza: I completely agree with you. I feel like this whole thing is a reaction to DA2's reception somehow, and Manveer is trying to make it look like we shouldn't listen to reviews and just buy games, which is crazy.

Get rid of the five stars for an entire year and see what happens. Why not? EVOLUTION MAN!

Interesting, but not. A journalist and a game developer arguing this issue is like a magazine editor and a journalist arguing about the reader. Y'all ain't making a lick of difference, and the arguments have all been heard before.

What's actually amusing is I don't know anyone personally who goes to Giantbomb and gives a shit about their reviews or review scores. Entertaining to read and informative? Sure, but all I need is a 30 minute quick look to judge if a game is right for me.

Review scores aren't as much a problem as a fascination with (and paying employees due to) review scores. Manveer has a "this or that" philosophy that is equally shitty on each side, while Patrick, and Giant Bomb in general, actually understands the happy medium balancing act. This all rings of someone scrambling to keep their job.

@sanchopanza said:

Am I the only one that thinks Manveer is talking out of his ass with the review vs. criticism thing? Surely you can't have one without the other...right? The review is there to point out the good things and criticise the bad things. If the reviewer finds the design of a game to be boring, isn't that as valid a criticism as saying the graphics suck balls?

What would a criticism free review even sound like? Every single such review would boil down to something like "when you push the button the guy moves, 5 stars".

No, you're not, and you've whittled down his argument perfectly. Subconsciouly or not, this may all be damage control for the scary-looking Mass Effect 3.

Anyone think this guy feels bad reviews incoming for Mass Effect 3?