On Games, Reviews, And Criticism -- Part 3

If the past few days has evidenced anything, it's that readers take reviews just as seriously as writers, developers and publishers. That's awfully important, too; ostensibly, you're what this is all about, right?

I've thoroughly enjoyed reading hundreds of comments from readers responding to my conversation with Mass Effect 3 senior designer Manveer Heir, who should be commended for throwing himself pie-first into the flames. There aren't many developers willing to take this kind of risk, but I'm hoping the great reaction from players means I'll be able to convince some other developers, too.

Maybe you're one of those developers. Drop me a line, won't you?

But that's getting ahead of ourselves. We still have a conversation to finish, in which Heir and I make our closing statements about where game reviews are, and where they're going.

If you're just joining us, catch up.

• On Games, Reviews, And Criticism -- Part 1
• On Games, Reviews, And Criticism -- Part 2

Note: This exchange took place over email, and I've done minimal editing to reflect the casual style.

--

Patrick,

Those are interesting points you bring up, and I have a hard time disagreeing with the examples and the points you brought up--so let me answer your question. Am I opposed to criticism being part of a review because it will skew more negatively? Well, that's just crazy--why would I be opposed for that reason? Except, I started to think about something you said in a previous email about how publishers have made reviews and their scores important, which has contributed to the whole problem...and then thought that maybe your question was spot-on.

See, when there is criticism in a review that seems unfair to me, and by unfair I don't mean the critic's opinion is wrong but rather that it is an unfair thing to hold against a game (linearity in Uncharted 3 or Modern Warfare 3, for example), that sucks. I realize the effect that has on the developer. In that regard, I think I do get upset at what feels like unfair reviews that contain criticism that should be pointed elsewhere. It's not that the criticism is negative, but rather that it is unfair (to me) to negatively hold that against a specific game, instead of holding it against, say, all games of that type. And it's unfair because this aggregate score, which I have been trained by all publishers ever to know as the value statement of my entire life and career, is affected, and I feel for those developers. It's hard to get away from that without removing the power from the score, which, as we discussed, is a whole huge mess.

As for the "it just needs more time" argument, if I am to buy that, I think we need to see improvements in reviews in the next year. I can't just take it on blind faith. Do game journalists have a conference where they discuss the state of the industry and ways to improve it, like what game developers have with GDC? If not, why not? That be one way to start getting forward thinking journalists to start pushing each other. I know having my peers challenge my ways of thinking and designing have made me better, so I have to imagine the same would be true for your field. How about we re-evaluate at the end of 2012, contingent on the Mayan predictions not being correct.

I will say that, in general, it seems like reviews are going down a bit now. I'm working on a sequel to a game that has a 96 rating and won Game of the Year awards all over the place in 2010. The Mass Effect 3 team will tell you that it is a better game than Mass Effect 2 in every single way. But there is no way, in my eyes, Mass Effect 3 is beating a 96 score because reviewers have started to get tougher (and rightly so). Scores are deflating slightly, and so maybe some of the pains and annoyances I am feeling are a result of that. I'm glad scores are going down--I think they should go down a lot more (hence the 5 star scale). But right now, it makes it seem like our games have less quality, and that is painful to those of us who spend our lives crafting experiences for players to enjoy. Instead of looking at these numbers as absolutes, maybe I should start looking at them like in sports: there are eras, and in video games we're in the "inflation era." Hopefully, a new era is upon us and we are just transitioning, which will probably coincide with new consoles.

But you bring up a good point that there is tons of criticism that does indeed belong inside a review. I think I've come around and agree with you there. So, then let me pose a final question to you and let you have the last word. Are there criticisms that are fair for a review and some that should be directed outside of a review? Is it fair to criticize a game for not being something it isn't trying to be, such as non-linear? Or are all criticisms always fair since, they are by their very nature, a matter of opinion?

Either way, I thank you for the very interesting conversation.

--

Manveer,

I'm not about to say anyone penning a review about Modern Warfare 3 or Uncharted 3 shouldn't use that platform to tear apart what they consider broken design. If a game doesn't work for the reviewer, they should say so. They can't play an expectations game. The fundamental difference you're alluding to is a point I touched upon earlier: the backseat designer. Those in the gaming press are often accused of being temporary writers as a convenient means to an end into games development. Hell, you admitted it was your path--it's not a new phenomenon. Look, it happens. Life changes, doors open, and the media business is brutal in its efficiency at producing tired cynics.

It's not a conspiracy. I'm not about to accuse every games writer of manipulating their occupation, but when a writer confuses their role as a critic with that of a designer, it becomes a problem. I'm guilty of this, as are most writers. Based on my conversations with developers, it's where a fundamental breakdown exists in the role of the critic, and what often inspires a negative reaction from creators. It's not our job to spend hundreds of words proselytizing about how a game might have been better if they had done this or that. Simply, it's our job to say why it did or didn't work and why--that's it.

I sympathize with your worries over the perception of Mass Effect 3, and you're not the first developer to do so. Giant Bomb's 2012 "The 'Check Yourself Before You Wriggety-Wreck Yourself' Award for Things That Need to 'Take a Break' Before They Become the 'Worst Trend'" was sequel fatigue. In our year-end discussions, we talked about where the fault could be attributed. Is it a Louis C.K. thing, where we're all complaining about things that are great? Or has an extended hardware generation, combined with the rising costs of AAA game development, and compounded by a downed economy, created an unhealthy situation for innovation?

The industry would do better to have a working standard for review scores, and I think we're naturally headed in that direction. Unfortunately, we don't have a "board" making decisions like that. Who would pick them? Who would follow them? But a benchmark solves avoidable issues for everyone. It's not even that everyone needs to be working with precisely the same rubric (though that would be awfully nice!), but take a look at your typical IGN review--they're still breaking games down int terms of presentation, graphics, sound, gameplay, and...lasting appeal? At least GameSpot's evolved, but I've worked with a bunch of ex-GameSpot for nearly a year now, and I still couldn't tell you what the hell a "tilt" used to be. Metacritic interprets the "intent" of each scale on its own, adding an unnecessary layer of mystery, and without a neutral outlet for developers and publications to talk with one another, tension rages.

Ultimately, that we're even having this conversation proves the value of criticism. Parkin wins! Or, rather, we both win. The real loser is, well, also us. We just don't have enough pieces of writing that have earned the criticism branding, and that's the reason there's such an uproar whenever a good one falls into our laps. It's not unlike the fervor that occurs when a game with exceptional storytelling appears. As fans of criticism, when someone makes a good point, we revel (excruciatingly) in every detail to a degree the original piece likely never intended or expected.

Also, when you say it out loud, it's pretty ridiculous the gaming media doesn't have an event to air its grievances, reflect and discuss. Such navel gazing is usually shoehorned into events like PAX, where an audience may show up to listen, but it's hardly the appropriate venue. Maybe I'll look into changing that.

Based on the way people have responded to this, I feel like we're onto something here. It's not often press and developers have an honest conversation outside of the traditional public relations process (it's usually in a bar after one-too-many drinks), and I hope to do more of these with you and other developers in the future.

I'll leave you with a piece of writing I'd like you to consider: Abbie Heppe's review of Metroid: Other M.

And I'll take you up on the offer to broach this topic again a year from now.

See you then,

Patrick

If the past few days has evidenced anything, it's that readers take reviews just as seriously as writers, developers and publishers. That's awfully important, too; ostensibly, you're what this is all about, right?

I've thoroughly enjoyed reading hundreds of comments from readers responding to my conversation with Mass Effect 3 senior designer Manveer Heir, who should be commended for throwing himself pie-first into the flames. There aren't many developers willing to take this kind of risk, but I'm hoping the great reaction from players means I'll be able to convince some other developers, too.

Maybe you're one of those developers. Drop me a line, won't you?

But that's getting ahead of ourselves. We still have a conversation to finish, in which Heir and I make our closing statements about where game reviews are, and where they're going.

If you're just joining us, catch up.

• On Games, Reviews, And Criticism -- Part 1
• On Games, Reviews, And Criticism -- Part 2

Note: This exchange took place over email, and I've done minimal editing to reflect the casual style.

--

Patrick,

Those are interesting points you bring up, and I have a hard time disagreeing with the examples and the points you brought up--so let me answer your question. Am I opposed to criticism being part of a review because it will skew more negatively? Well, that's just crazy--why would I be opposed for that reason? Except, I started to think about something you said in a previous email about how publishers have made reviews and their scores important, which has contributed to the whole problem...and then thought that maybe your question was spot-on.

See, when there is criticism in a review that seems unfair to me, and by unfair I don't mean the critic's opinion is wrong but rather that it is an unfair thing to hold against a game (linearity in Uncharted 3 or Modern Warfare 3, for example), that sucks. I realize the effect that has on the developer. In that regard, I think I do get upset at what feels like unfair reviews that contain criticism that should be pointed elsewhere. It's not that the criticism is negative, but rather that it is unfair (to me) to negatively hold that against a specific game, instead of holding it against, say, all games of that type. And it's unfair because this aggregate score, which I have been trained by all publishers ever to know as the value statement of my entire life and career, is affected, and I feel for those developers. It's hard to get away from that without removing the power from the score, which, as we discussed, is a whole huge mess.

As for the "it just needs more time" argument, if I am to buy that, I think we need to see improvements in reviews in the next year. I can't just take it on blind faith. Do game journalists have a conference where they discuss the state of the industry and ways to improve it, like what game developers have with GDC? If not, why not? That be one way to start getting forward thinking journalists to start pushing each other. I know having my peers challenge my ways of thinking and designing have made me better, so I have to imagine the same would be true for your field. How about we re-evaluate at the end of 2012, contingent on the Mayan predictions not being correct.

I will say that, in general, it seems like reviews are going down a bit now. I'm working on a sequel to a game that has a 96 rating and won Game of the Year awards all over the place in 2010. The Mass Effect 3 team will tell you that it is a better game than Mass Effect 2 in every single way. But there is no way, in my eyes, Mass Effect 3 is beating a 96 score because reviewers have started to get tougher (and rightly so). Scores are deflating slightly, and so maybe some of the pains and annoyances I am feeling are a result of that. I'm glad scores are going down--I think they should go down a lot more (hence the 5 star scale). But right now, it makes it seem like our games have less quality, and that is painful to those of us who spend our lives crafting experiences for players to enjoy. Instead of looking at these numbers as absolutes, maybe I should start looking at them like in sports: there are eras, and in video games we're in the "inflation era." Hopefully, a new era is upon us and we are just transitioning, which will probably coincide with new consoles.

But you bring up a good point that there is tons of criticism that does indeed belong inside a review. I think I've come around and agree with you there. So, then let me pose a final question to you and let you have the last word. Are there criticisms that are fair for a review and some that should be directed outside of a review? Is it fair to criticize a game for not being something it isn't trying to be, such as non-linear? Or are all criticisms always fair since, they are by their very nature, a matter of opinion?

Either way, I thank you for the very interesting conversation.

--

Manveer,

I'm not about to say anyone penning a review about Modern Warfare 3 or Uncharted 3 shouldn't use that platform to tear apart what they consider broken design. If a game doesn't work for the reviewer, they should say so. They can't play an expectations game. The fundamental difference you're alluding to is a point I touched upon earlier: the backseat designer. Those in the gaming press are often accused of being temporary writers as a convenient means to an end into games development. Hell, you admitted it was your path--it's not a new phenomenon. Look, it happens. Life changes, doors open, and the media business is brutal in its efficiency at producing tired cynics.

It's not a conspiracy. I'm not about to accuse every games writer of manipulating their occupation, but when a writer confuses their role as a critic with that of a designer, it becomes a problem. I'm guilty of this, as are most writers. Based on my conversations with developers, it's where a fundamental breakdown exists in the role of the critic, and what often inspires a negative reaction from creators. It's not our job to spend hundreds of words proselytizing about how a game might have been better if they had done this or that. Simply, it's our job to say why it did or didn't work and why--that's it.

I sympathize with your worries over the perception of Mass Effect 3, and you're not the first developer to do so. Giant Bomb's 2012 "The 'Check Yourself Before You Wriggety-Wreck Yourself' Award for Things That Need to 'Take a Break' Before They Become the 'Worst Trend'" was sequel fatigue. In our year-end discussions, we talked about where the fault could be attributed. Is it a Louis C.K. thing, where we're all complaining about things that are great? Or has an extended hardware generation, combined with the rising costs of AAA game development, and compounded by a downed economy, created an unhealthy situation for innovation?

The industry would do better to have a working standard for review scores, and I think we're naturally headed in that direction. Unfortunately, we don't have a "board" making decisions like that. Who would pick them? Who would follow them? But a benchmark solves avoidable issues for everyone. It's not even that everyone needs to be working with precisely the same rubric (though that would be awfully nice!), but take a look at your typical IGN review--they're still breaking games down int terms of presentation, graphics, sound, gameplay, and...lasting appeal? At least GameSpot's evolved, but I've worked with a bunch of ex-GameSpot for nearly a year now, and I still couldn't tell you what the hell a "tilt" used to be. Metacritic interprets the "intent" of each scale on its own, adding an unnecessary layer of mystery, and without a neutral outlet for developers and publications to talk with one another, tension rages.

Ultimately, that we're even having this conversation proves the value of criticism. Parkin wins! Or, rather, we both win. The real loser is, well, also us. We just don't have enough pieces of writing that have earned the criticism branding, and that's the reason there's such an uproar whenever a good one falls into our laps. It's not unlike the fervor that occurs when a game with exceptional storytelling appears. As fans of criticism, when someone makes a good point, we revel (excruciatingly) in every detail to a degree the original piece likely never intended or expected.

Also, when you say it out loud, it's pretty ridiculous the gaming media doesn't have an event to air its grievances, reflect and discuss. Such navel gazing is usually shoehorned into events like PAX, where an audience may show up to listen, but it's hardly the appropriate venue. Maybe I'll look into changing that.

Based on the way people have responded to this, I feel like we're onto something here. It's not often press and developers have an honest conversation outside of the traditional public relations process (it's usually in a bar after one-too-many drinks), and I hope to do more of these with you and other developers in the future.

I'll leave you with a piece of writing I'd like you to consider: Abbie Heppe's review of Metroid: Other M.

And I'll take you up on the offer to broach this topic again a year from now.

See you then,

Patrick

Great article guys...

Excellent

Kleptok is all up on this piece!

I am really getting more and more tired of people quoting that Louis C.K. thing.

That was a great conclusion. I think Manveer (beyond having a fantastic name) raises some interesting points, particularly concerning reviews being used as a critical soapbox.

Also, the idea of some form of industry meeting is a very good one. Who knows, maybe Whiskey will wind up setting one up one day?

Game journalists love to navel gaze. As an avid game site visitor and podcast listener for years, I can say I am WAY over this topic.

Just write, if it's interesting, I'll read it. It's not that fucking complicated. How many times are we going to have to rehash this, frankly boring and meaningless, story?

@DFSVegas said:

Game journalists love to navel gaze. As an avid game site visitor and podcast listener for years, I can say I am WAY over this topic.

Just write, if it's interesting, I'll read it. It's not that fucking complicated. How many times are we going to have to rehash this, frankly boring and meaningless, story?

I'm with you on that, which is why I'd hoped to avoid that somewhat by involving a developer directly.

whoever gives ME3 below 5/5 is Literally Hitler

Games reviews should contain criticism but should only render a "judgement" on that criticism relative to other similar experiences.

The value of true criticism is found in its ability to shape our understanding of a product with respect to the whole medium.

I believe criticism should not be pejorative or effusive, whereas with reviews I have no such qualms.

I think when Manveer had issue with criticism finding its way into a review he was kind of missing the point. Reviews aren't done in a vacuum. The thrill of something new and exciting can't be feigned. If Uncharted 3 wore out some people because the linear action has been overdone a little bit, what is the reviewer supposed to do?

Reviews are purchasing advice written by other people, what is it exactly he's giving advice on if it isn't the experience he had with the game? I would hope the reviewer was being honest if a game gave him/her a sensation that they were retreading a lot of old tricks and paths from the last game.

While yes it sucks that Uncharted 3 got hit by a criticism about linear game, that's another thing reviews can do. The time and place a game is released in needs to be taken into account. Uncharted 3 got docked because it was released in a year heavy with sequels and the consoles coming past their 5 year life cycle.

Uncharted 2 reveled into showing us what this generation of consoles can really do, it was an eye-opener and the scores it got reflected that. So why is it all of a sudden unfair 2 years later to say, "hey the spark isn't really there this time" with Uncharted 3?

You can't have it both ways.

@fusrodah said:

whoever gives ME3 below 5/5 is Literally Hitler

I can guarantee there will be some 80s and 90s. Just the way the industry has so many different personalities.

@patrickklepek said:

@DFSVegas said:

Game journalists love to navel gaze. As an avid game site visitor and podcast listener for years, I can say I am WAY over this topic.

Just write, if it's interesting, I'll read it. It's not that fucking complicated. How many times are we going to have to rehash this, frankly boring and meaningless, story?

I'm with you on that, which is why I'd hoped to avoid that somewhat by involving a developer directly.

It's still boring, and masturbatory.

I'm in the same exact boat as @DFSVegas. The only reason I can see for this article existing, is that if Mass Effect 3 ends up getting less than expected scores, this is the article that everyone will link to when discussing the internet hiccup that that will create. That situation of course being completely trite and totally uninteresting, as well.

I like a lot of what you do Patrick, but these invented news articles are total garbage. You might as well run all the rumor articles that the the other sites do, it's just as wrote.

Great read again

Really enjoyed these. Hope to see more discussion like this with others in the industry.

Alex's 2-paragraph article on the Max Payne 3 release date slipping was more interesting and entertaining than this 3-part series.

Great stuff Patrick, thank you and Manveer for sharing it with us. It was rather insightful, and very encouraging to see that the people 'on the other side' are as aware of the issues surrounding game reviewing as the rest of us are.

Thanks again, and I really hope other game developers are as open to having another conversation of this calibre with you.

I enjoyed this series, I'd enjoy seeing some roundtable discussion videos like this with journos and devs shooting the shit in a more informal context.

But you bring up a good point that there is tons of criticism that does indeed belong inside a review. I think I've come around and agree with you there. So, then let me pose a final question to you and let you have the last word. Are there criticisms that are fair for a review and some that should be directed outside of a review? Is it fair to criticize a game for not being something it isn't trying to be, such as non-linear? Or are all criticisms always fair since, they are by their very nature, a matter of opinion?

To a point, yes. If ME3 is a linear experience and it actually hurts the quality of the experience, then it should be mentioned in the review. The gaming landscape is constantly shifting. Even more so when you have blockbuster games like Skyrim coming out. You bet that ME3 is going to be compared to Skyrim. It might even suffer from the lack of any real exploration or the fact that it's a more linear experience than skyrim.

And that's totally fair by my standards. You cant review games in a vacuum, it's simply not possible.

Then, there is the fact that the situation with gaming "reviews" is really messed up. When scores matter for bonuses and when money is involved, I see no easy way out. And really, that's why I would never really expect a developer to be 100% fine with reviews. It's really hard to not care when part of your paycheck depends on the score given by a bunch of humans.

And really, great story Patrick. That's why I love GiantBomb.

I think there is something to be said for having a writers conference for game reviewers. I work for an alternative newsweekly, and we have workshops and conferences throughout the year to soundboard, generate ideas, and critique both individual pieces through workshop, and critique industry troubles at large in a wide conference setting. It helps not only weed out bad habits and traits that plague certain aspects of the industry, but also helps share good work practices with other organizations. That kind of dialogue will also help in cutting away the fat and move the industry toward a more standardized rating system.

Please try and set up some sort of game journalists meeting where shit like this gets figured out, as suggested.

I enjoyed this series.

Game journalists love to navel gaze. As an avid game site visitor and podcast listener for years, I can say I am WAY over this topic.

Just write, if it's interesting, I'll read it. It's not that fucking complicated. How many times are we going to have to rehash this, frankly boring and meaningless, story?

It's still boring, and masturbatory.

I'm in the same exact boat as @DFSVegas. The only reason I can see for this article existing, is that if Mass Effect 3 ends up getting less than expected scores, this is the article that everyone will link to when discussing the internet hiccup that that will create. That situation of course being completely trite and totally uninteresting, as well.

I like a lot of what you do Patrick, but these invented news articles are total garbage. You might as well run all the rumor articles that the the other sites do, it's just as wrote.

Alex's 2-paragraph article on the Max Payne 3 release date slipping was more interesting and entertaining than this 3-part series.

@Make_Me_Mad said:

I am really getting more and more tired of people quoting that Louis C.K. thing.

Hey look at this guy, complaining about an awesome joke that Louis C.K. made. Can no one be happy? ;)

Fantastic piece, and great writing from both sides.

I've written for TouchGen.com since it's launch when the App Store was introduced, and we have always treated our reviews very seriously as BOTH critiques and reviews. Critical analysis doesn't always just involve negative observations, even though the word "critic" usually has that connotation. Have I sometimes felt like a friggin beta tester? Hell yes, but I can't tell you how many times, after publishing a review, I have received an email from a developer thanking me for being honest and in-depth in my analysis. This is not just for games that received a good score, and it shows that when I developer reads a review that actually offers solutions to some of the issues in their game, they can take that seriously to help improve the player experience in the future.

I like this part of reviewing games, although I'm starting to realize that perhaps I have too much critiquing in my reviewing? What a crazy thing to think about.

I experienced "bad score fallout" a couple years ago when a wrote the review for Modern Combat (the original Gameloft CoD clone). The amount of stupidity in the comments was phenomenal, and really showed me the state of many current review readers. I gave the game a generous 3.5/5 (looking back, it probably should have been a 3) and people were outraged by a score that is listed as "good" in our rating descriptions. They also played the usual part of comparing my review to completely different games that received higher scores, as if that meant anything. It was obvious that the majority reviews focused on the score, instead of reading my in-depth analysis of why the gaming wasn't amazing. I don't see any way around this other than to totally abolish total scores, which as you guys both said, simply won't happen. :/

@DTKT said:

But you bring up a good point that there is tons of criticism that does indeed belong inside a review. I think I've come around and agree with you there. So, then let me pose a final question to you and let you have the last word. Are there criticisms that are fair for a review and some that should be directed outside of a review? Is it fair to criticize a game for not being something it isn't trying to be, such as non-linear? Or are all criticisms always fair since, they are by their very nature, a matter of opinion?

To a point, yes. If ME3 is a linear experience and it actually hurts the quality of the experience, then it should be mentioned in the review. The gaming landscape is constantly shifting. Even more so when you have blockbuster games like Skyrim coming out. You bet that ME3 is going to be compared to Skyrim. It might even suffer from the lack of any real exploration or the fact that it's a more linear experience than skyrim.

And that's totally fair by my standards. You cant review games in a vacuum, it's simply not possible.

That's dumb to say. I don't know how else to phrase it, but that's dumb. To compare two games isn't wrong, in itself, and it makes a lot of sense sometimes. With the smattering of FPS games coming out, it makes sense to compare Battlefield 3 and MW3. Even if they're not enitrely the same offering, they have similar mechanics and are worth comparing and contrasting in terms of their design, their feel and the overall value of the package. They exist in the same bubble: modern first-person shooters with a heavy emphasis on multiplayer.

However, to compare two totally separate games and write it off as 'acceptable'? To compare a third-person sci-fi action/RPG driven by story with a primarily first-person RPG set in a fantasy sandbox? The games are not trying to be similar; Mass Effect isn't trying to ride on the coattail of Skyrim, or even attempting to compete in the same space as it. Mass Effect is being aimed at an entirely different audience.

In a review, if a reviewer states that Mass Effect 3 sucks because it isn't as open as Skyrim, the skills don't progress the same as Skyrim, there are no fucking dragons like in Skyrim, that's a review that you should not listen to. If two games that are trying to accomplish two different things are put side-by-side in a review, then that means that the reviewer is playing the wrong game. You can't compare Mass Effect and Skyrim the same way you can't compare Sudoku and a Rubik's Cube. They have similar elements and, when boiled down, can both be classified as the same thing, but they're not being marketed to the same group. Sudoku is a number puzzle and a Rubik's Cube is a spatial/colour puzzle, just as Skyrim and Mass effect, while both RPGs, cater to wholly different audiences. As a fan of both Skyrim and Mass Effect, I do not to go Skyrim to form a crew to whom I become emotionally attached to, and I don't go to Mass Effect to wander around for hours and work on my stealth skill.

To even suggest that Mass Effect is going to be, or should be, compared to Skyrim is brainless. You can't compare games just because one game did new things that push the genre forward, because the other game might not benefit at all from including those things. It doesn't make sense, and it shouldn't happen without solid reasoning.

I'm loving this. More games criticism please!!

@dvorak said:

@patrickklepek said:

@DFSVegas said:

Game journalists love to navel gaze. As an avid game site visitor and podcast listener for years, I can say I am WAY over this topic.

Just write, if it's interesting, I'll read it. It's not that fucking complicated. How many times are we going to have to rehash this, frankly boring and meaningless, story?

I'm with you on that, which is why I'd hoped to avoid that somewhat by involving a developer directly.

It's still boring, and masturbatory.

I'm in the same exact boat as @DFSVegas. The only reason I can see for this article existing, is that if Mass Effect 3 ends up getting less than expected scores, this is the article that everyone will link to when discussing the internet hiccup that that will create. That situation of course being completely trite and totally uninteresting, as well.

I like a lot of what you do Patrick, but these invented news articles are total garbage. You might as well run all the rumor articles that the the other sites do, it's just as wrote.

I think that's a bit harsh to be honest. I share some of @DFSVegas sentiments, but even I felt like this series was interesting because of how open the dialogue was. Instead of what we usually get - one off rants on some developers blog, or a series of angry tweets from a game jurno - this was an honest discussion between both sides of the issue. I'm all for this series continuing with different developers and hopefully different members of the Giant Bomb staff as well. Bravo Patrick and Manveer.

I'll agree it's a topic that has been done to death, and it is a discussion that has been going on for several years at this point. The difference with this piece is that it brought something new to the table. I'd also add that I believe the reason it continues to come up for discussion is not just for the navel gazing (although that does play a part) but that there are legitimate issues to be worked out. It is a conversation that still needs to happen. Given a choice, I'd rather the discussion be interesting to read and constructive towards a solution like this series has been than what we usually get.

Awesome Dialogue. More of this!!!

Patrick and Manveer, this was an amazing read. Thank you so much for your courage, modesty and insight. This sfuff you're doing right here, that's showing the grounding stone for what could be new road to walk on.

@patrickklepek: Yeah, to be fair, it is a different twist on the topic. I can't lie, I do like getting an insiders view. As a guy who loved the Bastion coverage, I'd be a tad bit hypocritical to say I see no value in this specific piece.

We really do need to get over this topic though. I know Gaming is a relatively young industry, and a lot of young, ambitious people work in said industry. It does seem weird to see journalists write about journalism though. Maybe it's me? I don't know.

I guess the way I see it, the answer to this issue is just time. Gamers are growing older, but gaming is still a young person's activity. As the people surrounding the industry start to mature, I think the quality of everything will increase. The games, the stuff we write about games, and the people who enjoy both of those things.

Not trying to be dismissive, I just have an immediate reaction to seeing journalists journalising about journalism. It's kind of the only place you really see it.

I think this last part really tied it all together nicely and it's interesting to see perspectives shift for both of you with new ideas. It's refreshing to see opposite sides discuss without necessarily working too hard to change the other ones perspective, just exchanging of ideas and thoughts.

I like how he's worried about the scores for Mass Effect 3. Yeah, it's not going to get a better score than ME2 unless you do something NEW. That's how reviews work in all media, not just video games. So do something new and better with your game. Don't copy ME2 and make everything slightly better, and then say "But it's the same game only slightly better! It should get a slightly better score!" No, it shouldn't. Because I can get that game for 20 bucks, only it's slightly not as good as this one. Are you going to charge 21 dollars for the sequel? No? Then don't worry about scores.

Another factor that I didn't see directly addressed (but which I feel was implied to a degree when talking about criticism in games) is the human factor. No two reviewers are the same despite them liking games in the same genre. Reviews are very much subjective even though they should be written as objectively as possible. Reviewers get blasted ALL THE TIME in regards to this and we see it all the time from forum trolls that accuse said reviewer to be biased -- by either showering a game with too much praise or condemning it to hell -- the reviewer has to constantly walk a razor's edge fully knowing that you can please some people some of the time and none of the people all of the time (to paraphrase Abraham Lincoln).

I think readers should also take their share of blame in this debate as well for not taking into account that reviewers are human with opinions that might not be the same as theirs, but that doesn't mean they are biased or have a secret agenda when Game X doesn't get that 9.5 the Reader had already convinced himself in their mind it would get. Some people get WAY too defensive over certain games as if they had a personal stake in its development, and that's just foolish.

The Bombcasts from PAX and E3 and such, with the room full of various developers and writers, are my favorites of the year, because we get to hear opinions on the industry from lots of people from lots of places. While most of the regular podcasts aren't really worth too much after a month or two, because the games being previewed have come out, the expo shows can be listened to for years, since they tend to discuss ongoing concerns in the industry. That seems like the place to bring this topic up again, with other writers of reviews and developers discussing what they want from reviews.

Fantastic article guys

I think i'm done with reading about the whole "review" debacle.

If developers and publishers stopped clinging so desperately to that metacritic score like it's a objective measure of a game's worth, all would be right with the world. Metacritic is a gathering pool of opinions and meaningless scores that every website treats differently. If every site and reviewer would have the exact same opinions and judge games by the exact same metric, then your score would actually mean something.

Look at how he presents himself:

I'm working on a sequel to a game that has a 96 rating and won Game of the Year awards all over the place in 2010.

If he would just disregard the score, and actually take some criticism to heart instead of getting high of the fumes on the metacritic average, then i'd be satisfied.

I'm not saying they didn't do that with ME3, but I just hate developers that have to point to the metacritic average to justify their position in the industry these days.

@The_Nubster said:

@DTKT said:

But you bring up a good point that there is tons of criticism that does indeed belong inside a review. I think I've come around and agree with you there. So, then let me pose a final question to you and let you have the last word. Are there criticisms that are fair for a review and some that should be directed outside of a review? Is it fair to criticize a game for not being something it isn't trying to be, such as non-linear? Or are all criticisms always fair since, they are by their very nature, a matter of opinion?

To a point, yes. If ME3 is a linear experience and it actually hurts the quality of the experience, then it should be mentioned in the review. The gaming landscape is constantly shifting. Even more so when you have blockbuster games like Skyrim coming out. You bet that ME3 is going to be compared to Skyrim. It might even suffer from the lack of any real exploration or the fact that it's a more linear experience than skyrim.

And that's totally fair by my standards. You cant review games in a vacuum, it's simply not possible.

That's dumb to say. I don't know how else to phrase it, but that's dumb. To compare two games isn't wrong, in itself, and it makes a lot of sense sometimes. With the smattering of FPS games coming out, it makes sense to compare Battlefield 3 and MW3. Even if they're not enitrely the same offering, they have similar mechanics and are worth comparing and contrasting in terms of their design, their feel and the overall value of the package. They exist in the same bubble: modern first-person shooters with a heavy emphasis on multiplayer.

However, to compare two totally separate games and write it off as 'acceptable'? To compare a third-person sci-fi action/RPG driven by story with a primarily first-person RPG set in a fantasy sandbox? The games are not trying to be similar; Mass Effect isn't trying to ride on the coattail of Skyrim, or even attempting to compete in the same space as it. Mass Effect is being aimed at an entirely different audience.

In a review, if a reviewer states that Mass Effect 3 sucks because it isn't as open as Skyrim, the skills don't progress the same as Skyrim, there are no fucking dragons like in Skyrim, that's a review that you should not listen to. If two games that are trying to accomplish two different things are put side-by-side in a review, then that means that the reviewer is playing the wrong game. You can't compare Mass Effect and Skyrim the same way you can't compare Sudoku and a Rubik's Cube. They have similar elements and, when boiled down, can both be classified as the same thing, but they're not being marketed to the same group. Sudoku is a number puzzle and a Rubik's Cube is a spatial/colour puzzle, just as Skyrim and Mass effect, while both RPGs, cater to wholly different audiences. As a fan of both Skyrim and Mass Effect, I do not to go Skyrim to form a crew to whom I become emotionally attached to, and I don't go to Mass Effect to wander around for hours and work on my stealth skill.

To even suggest that Mass Effect is going to be, or should be, compared to Skyrim is brainless. You can't compare games just because one game did new things that push the genre forward, because the other game might not benefit at all from including those things. It doesn't make sense, and it shouldn't happen without solid reasoning.

Damn, you jumped on that hard.

Expectations always change. That's what I meant to say. If the Skyrim comes out and is amazing, ME3 is bound to be seen in a different way. You are right, it might not be "fair" nor even a valid comparison, but it will happen.

Kudos to Manveer for being willing to do this. This is one of the most interesting articles I've read anywhere in ages.

I think it's real shame when a review that has some healthy criticism in it, just achieves abuse from fanboys. I agree that every now and again the critisisms are groundless, but overall, I think that critical reviews are constructive.

I really like this segment and enjoy reading both patricks thoughts on the subject and also manveers. Its refreshing to get a fully sided discussion that is not only interesting to read but also insightful. Us as consumers rarely get to see the working behind reviews that we read and the thought processes behind them. Personally i'm of the opinion that criticism has a place in reviews, but should not be the focal point. When i read a review the main points i am looking at are how the game performs, if like in uncharted the linearity detracts from the experience then yes it should be in the review but thats where it should end. I wouldn't want read a review of fifa 12 and the reviewer likes the game but sends the article talking about the necessity of annualized sports games.

All in all love the article and love the site.

I think it's unfair to claim that if linearity is criticized in Uncharted, in must be criticized in MW3. Uncharted is an adventure game, and an uncomfortable sense of linearity could easily dampen the experience for a reviewer. That's not so in MW3.

Man I love this kind of stuff. Anything that pushes my brain and makes my ego step out for a minute for me to contemplate is awesome. The mind blowing part about all this is adding it together. Reviews are primarily read by people who have already made their buying decision. A number is assigned to the review by the reviewer. Then a publisher either gives a bonus to a developer or fires them based on the review number.

I feel bad for Uncharted 3 taking all the heat for this. It would be pretty sweet to see the highest review score be a 90 equivalent in 2012.

Do developers solely get bonuses (and personal satisfaction) depending on the critics' scores? Are sales numbers ultimately not more important? Not only for the managers who decide how much to pay the makers of the games, but also for those makers themselves. I know I find it more satisfying, if more people than ever bought a new Volkswagen (where I work) even though critics were pessimistic.

So who cares whether or not Mass Effect 3 gets a lower rating than 96, as long as more gamers will buy that - I'm certain - fantastic game?!

The problem I had with the original review by Parkin that spawned this whole debate was that you could sum up his entire review in this one sentence "Uncharted 3 is the most exciting linear game of all time, and is impeccably crafted, but I don't think games should be linear. 8/10"

Someone coming to Eurogamer for a review of the next entry in a series of linear action games isn't going to find that review at all helpful. If you are thinking of buying Uncharted 3 and know anything about the series you're expecting a linear action game. Eurogamer should have handed the review to someone who was interested in a linear action game and had Mr. Parkin write an editorial on why he thinks games should be open and inherently involve player choice and consequences.

As an example, does giving a horror game to a reviewer who doesn't like horror games or movies make sense? I don't. I think that's what they did over at Eurogamer.

In other words, I agree with Manveer's original thoughts entirely.

Does criticism have to be "fair" to be valid? Interesting question. I would lean towards...no. The idea of "fairness" is pretty subjective. Probably depends on how well the critic backs their arguments, right? It's amazing how much a well reasoned argument can change someones definition of fair.

Should broad criticisms that could be addressed to a whole generation of games be included as part of a review of one game? That seems to be the question Manveer is worried about. But let me put it another way... should a reviewer NOT bring up an issue just because other games are guilty of it?

Liked the point about reviewers falling into the trap of being backseat designers too.

Discussion overload.

i enjoyed this series also...very enjoyable.. saw not long after another website had a similar story up online. not as intense..just paragraph comments from various developers

I must say that i really dont understand the logic in the following statement made by Manweer:

Start of quote

" I will say that, in general, it seems like reviews are going down a bit now. I'm working on a sequel to a game that has a 96 rating and won Game of the Year awards all over the place in 2010. The Mass Effect 3 team will tell you that it is a better game than Mass Effect 2 in every single way. But there is no way, in my eyes, Mass Effect 3 is beating a 96 score because reviewers have started to get tougher (and rightly so)."

End of quote

So according to Manveer review scores are generally lower these days than they were before. Then he makes a statement on how Mass Effect 2 has a 96 rating (presumably on Metacritic) and after that he lets us know that the game has won numerous Game of the year awards.

Now comes the really puzzling bit as according to Manveer the Mass Effect 3 team thinks that Mass Effect 3 will be a better game than Mass Effect 2 "in every single way" ( I would love to find a development team that will publicly say that the game that they are developing is a worse game than the one before it).

Then Manveer says that there is no way that Mass Effect 3 will beat the 96 score as according to him "reviewers have started to get tougher". So the only reason that Mass Effect 3 might get a lower Metacritic score than Mass Effect 2 is that game reviewers dont give so easily good review scores anymore?

There could be another reason for a lower score for Mass Effect 3 but apparently that reason isnt obivous to Manveer. The game might just not be as good as Mass Effect 2 (Regardless of what the development team thinks personally of the quality of Mass Effect 3).

Also as I suspected Manveer doesnt mention at all the influence that game publishers have on gaming sites with their ability to pull advertisements from the sites and of their ability to cut access to games before release. One only has to think what happened to a certain co-founder of Giant Bomb when he was working at Gamespot and gave a review that wasnt favorable to a certain game.

But apparently the only problem in reviewing games is the tendency of the readers to rely on review scores and seemingly nothing else.

Games designers feel that their games shouldn't get low scores. gotcha.

@damnable_fiend said:

Games designers feel that their games shouldn't get low scores. gotcha.

That is literally the simplest and most unintelligent way you could have possibly understood that article. Well done.

@Archaen: Just like the nature of reviews, reader's reaction to reviews are entirely subjective.

When I read the review, I didn't interpret his criticism of UC3 as a fundamental mismatch between his expectations and the game's linear conventions. What I saw in Parkin's review was the idea that - yes UC3 is well produced and more often than not of excellent quality, but the linearity of the game occasionally worked against the player's experience.

Parkin wasn't alone on this - I read other reviews (and it came up a couple times in discussions in Bombcasts) where critics, while acknowledging that UC3 was a mostly great game, also noted that it suffered gameplay hiccups (instant deaths, unclear paths) because of its rigid linearity.

It's not that Parkin was refuting linearity in games - it's a case of him pointing out that the balance between player choice and linearity had been tipped in UC3 in such a way that it led to more frustrating moments - it's likely one of the reasons why as great a game as UC3 is - it wasn't as well received as UC2.