On Games, Reviews, And Criticism -- Part 3

If the past few days has evidenced anything, it's that readers take reviews just as seriously as writers, developers and publishers. That's awfully important, too; ostensibly, you're what this is all about, right?

I've thoroughly enjoyed reading hundreds of comments from readers responding to my conversation with Mass Effect 3 senior designer Manveer Heir, who should be commended for throwing himself pie-first into the flames. There aren't many developers willing to take this kind of risk, but I'm hoping the great reaction from players means I'll be able to convince some other developers, too.

Maybe you're one of those developers. Drop me a line, won't you?

But that's getting ahead of ourselves. We still have a conversation to finish, in which Heir and I make our closing statements about where game reviews are, and where they're going.

If you're just joining us, catch up.

• On Games, Reviews, And Criticism -- Part 1
• On Games, Reviews, And Criticism -- Part 2

Note: This exchange took place over email, and I've done minimal editing to reflect the casual style.

--

Patrick,

Those are interesting points you bring up, and I have a hard time disagreeing with the examples and the points you brought up--so let me answer your question. Am I opposed to criticism being part of a review because it will skew more negatively? Well, that's just crazy--why would I be opposed for that reason? Except, I started to think about something you said in a previous email about how publishers have made reviews and their scores important, which has contributed to the whole problem...and then thought that maybe your question was spot-on.

See, when there is criticism in a review that seems unfair to me, and by unfair I don't mean the critic's opinion is wrong but rather that it is an unfair thing to hold against a game (linearity in Uncharted 3 or Modern Warfare 3, for example), that sucks. I realize the effect that has on the developer. In that regard, I think I do get upset at what feels like unfair reviews that contain criticism that should be pointed elsewhere. It's not that the criticism is negative, but rather that it is unfair (to me) to negatively hold that against a specific game, instead of holding it against, say, all games of that type. And it's unfair because this aggregate score, which I have been trained by all publishers ever to know as the value statement of my entire life and career, is affected, and I feel for those developers. It's hard to get away from that without removing the power from the score, which, as we discussed, is a whole huge mess.

The latest Batman was once again beloved, but will Rocksteady get that treatment a third time?

As for the "it just needs more time" argument, if I am to buy that, I think we need to see improvements in reviews in the next year. I can't just take it on blind faith. Do game journalists have a conference where they discuss the state of the industry and ways to improve it, like what game developers have with GDC? If not, why not? That be one way to start getting forward thinking journalists to start pushing each other. I know having my peers challenge my ways of thinking and designing have made me better, so I have to imagine the same would be true for your field. How about we re-evaluate at the end of 2012, contingent on the Mayan predictions not being correct.

I will say that, in general, it seems like reviews are going down a bit now. I'm working on a sequel to a game that has a 96 rating and won Game of the Year awards all over the place in 2010. The Mass Effect 3 team will tell you that it is a better game than Mass Effect 2 in every single way. But there is no way, in my eyes, Mass Effect 3 is beating a 96 score because reviewers have started to get tougher (and rightly so). Scores are deflating slightly, and so maybe some of the pains and annoyances I am feeling are a result of that. I'm glad scores are going down--I think they should go down a lot more (hence the 5 star scale). But right now, it makes it seem like our games have less quality, and that is painful to those of us who spend our lives crafting experiences for players to enjoy. Instead of looking at these numbers as absolutes, maybe I should start looking at them like in sports: there are eras, and in video games we're in the "inflation era." Hopefully, a new era is upon us and we are just transitioning, which will probably coincide with new consoles.

But you bring up a good point that there is tons of criticism that does indeed belong inside a review. I think I've come around and agree with you there. So, then let me pose a final question to you and let you have the last word. Are there criticisms that are fair for a review and some that should be directed outside of a review? Is it fair to criticize a game for not being something it isn't trying to be, such as non-linear? Or are all criticisms always fair since, they are by their very nature, a matter of opinion?

Either way, I thank you for the very interesting conversation.

Gears of War 3 is the definition of careful iteration, a concept that's caused a split amongst critics.

--

Manveer,

I'm not about to say anyone penning a review about Modern Warfare 3 or Uncharted 3 shouldn't use that platform to tear apart what they consider broken design. If a game doesn't work for the reviewer, they should say so. They can't play an expectations game. The fundamental difference you're alluding to is a point I touched upon earlier: the backseat designer. Those in the gaming press are often accused of being temporary writers as a convenient means to an end into games development. Hell, you admitted it was your path--it's not a new phenomenon. Look, it happens. Life changes, doors open, and the media business is brutal in its efficiency at producing tired cynics.

Mortal Kombat showed what happens when a developer reboots without losings its roots.

It's not a conspiracy. I'm not about to accuse every games writer of manipulating their occupation, but when a writer confuses their role as a critic with that of a designer, it becomes a problem. I'm guilty of this, as are most writers. Based on my conversations with developers, it's where a fundamental breakdown exists in the role of the critic, and what often inspires a negative reaction from creators. It's not our job to spend hundreds of words proselytizing about how a game might have been better if they had done this or that. Simply, it's our job to say why it did or didn't work and why--that's it.

I sympathize with your worries over the perception of Mass Effect 3, and you're not the first developer to do so. Giant Bomb's 2012 "The 'Check Yourself Before You Wriggety-Wreck Yourself' Award for Things That Need to 'Take a Break' Before They Become the 'Worst Trend'" was sequel fatigue. In our year-end discussions, we talked about where the fault could be attributed. Is it a Louis C.K. thing, where we're all complaining about things that are great? Or has an extended hardware generation, combined with the rising costs of AAA game development, and compounded by a downed economy, created an unhealthy situation for innovation?

The industry would do better to have a working standard for review scores, and I think we're naturally headed in that direction. Unfortunately, we don't have a "board" making decisions like that. Who would pick them? Who would follow them? But a benchmark solves avoidable issues for everyone. It's not even that everyone needs to be working with precisely the same rubric (though that would be awfully nice!), but take a look at your typical IGN review--they're still breaking games down int terms of presentation, graphics, sound, gameplay, and...lasting appeal? At least GameSpot's evolved, but I've worked with a bunch of ex-GameSpot for nearly a year now, and I still couldn't tell you what the hell a "tilt" used to be. Metacritic interprets the "intent" of each scale on its own, adding an unnecessary layer of mystery, and without a neutral outlet for developers and publications to talk with one another, tension rages.

Ultimately, that we're even having this conversation proves the value of criticism. Parkin wins! Or, rather, we both win. The real loser is, well, also us. We just don't have enough pieces of writing that have earned the criticism branding, and that's the reason there's such an uproar whenever a good one falls into our laps. It's not unlike the fervor that occurs when a game with exceptional storytelling appears. As fans of criticism, when someone makes a good point, we revel (excruciatingly) in every detail to a degree the original piece likely never intended or expected.

Abbie Heppe came under intense scrutiny for her review of Metroid: Other M. What do you think?

Also, when you say it out loud, it's pretty ridiculous the gaming media doesn't have an event to air its grievances, reflect and discuss. Such navel gazing is usually shoehorned into events like PAX, where an audience may show up to listen, but it's hardly the appropriate venue. Maybe I'll look into changing that.

Based on the way people have responded to this, I feel like we're onto something here. It's not often press and developers have an honest conversation outside of the traditional public relations process (it's usually in a bar after one-too-many drinks), and I hope to do more of these with you and other developers in the future.

I'll leave you with a piece of writing I'd like you to consider: Abbie Heppe's review of Metroid: Other M.

And I'll take you up on the offer to broach this topic again a year from now.

See you then,

Patrick