Posted by MrCHUP0N (246 posts) -

Are a game's positive reviews rendered moot because they were written right when the game in question was released? If you think this is a ridiculous sentiment, hold on just a bit while I recount my experience a few days ago. I was in my local Gamestop on Sunday, picking up a long-overdue copy of Advance Wars: Days of Ruin. Behind me in line was a couple waiting to pay for Fallout 3 for the Xbox 360. The woman remarked, "Wow, so I heard this game got really good reviews." The man replied, almost without hesitation, "Yeah, but that's because it just came out."

The first thought to pop into my head was, "0_o" -- but I told myself to listen on in case the guy had a follow-up. "Everybody gets caught up in the excitement," he explained to the woman, "so everyone's really looking forward to playing this new thing and they're happy to finally get their hands on it." She replied, "Ah. So it probably makes more sense to wait and buy it later when people start to see what might be wrong with it."

As you can probably imagine, "o_0" was still lingering in my head. Now, it's not as if what they were saying was completely devoid of merit, but at least on the surface it's plain to see that there were a few incorrect assumptions being made about what (good) reviewers do and how they come up with the evaluations that they do. What could drive the belief that a highly positive review is based on its proximity to the game's release?

The Hype Factor: "Everybody gets caught up in the excitement"

It certainly feels good to get your hands on a highly-anticipated game for the first time, and for some critics, the early access makes it even better. I argue, however, that professional critics don't get to the position that they're in by falling heads-over-heels for every new hot title. Part of the job is being able to look at every product from as objective an angle as possible. The critic is telling you whether or not to spend not only hours of your life but also SIXTY DOLLARS on the game in question -- and quite frankly, I would personally rather debate with someone accusing me of low-balling a game than feel guilty about having someone upset at spending time and money on a game he ultimately didn't enjoy. That's not to say I approach games with the intention of being overly critical, but rather I approach with caution.

Taking it one step further...

Covering All Your Bases: "It probably makes more sense to wait [for] when people start to see what might be wrong with [the game]."

This is a difficult one to pin down, but first let's address what's potentially wrong with this thinking. If we're being cynical and bitter, it's easy to say that this statement is made under the assumption that game reviewers don't thoroughly play the game in question. After all, what's the logic behind saying that one won't know what's wrong with a game until it's been out for a few months? (I can hear, "Yeah, but..." coming -- I'll get to you, don't worry.)

A reviewer generally plays a game from start to finish, or at the very least to an extent where an educated opinion can be made (to me, that means "finished"). A role-playing game isn't abandoned after only half of the adventure has been complete. A fighting game isn't dropped after playing in a couple of scattered bouts. A reviewer should be as complete as possible with the game in question, not only to make sure that as many issues as possible have been uncovered but also to make sure that any residual excitement from hype -- for those prone to such a thing -- has died down. In my experience, this is surely the case.

It's storytime again, and I promise you it'll be short. Let's jump back to erstwhile Associate Editor Alex Navarro, now at Harmonix. When Guitar Hero III launched, the man posted his review after the product launch all because he wanted to give you an accurate and thorough representation of how online worked against real competition. Hell -- the review code's online play was buggy, so he lugged his butt to Best Buy, picked up a retail copy of Guitar Hero III for each console, and hauled the plastic motherlode back to his reviewing dungeon over the weekend just to make sure he could get in some quality time.

So, yes -- good reviewers have been and will continue to be thorough and complete in assessing a game. No matter how long a game has been in the market for, poor design decisions, planning and ideas won't go away, or suddenly appear out of thin air, or get any worse or better (put your hand down -- I'm getting there). For the most part, to say that a game would review better with time is insinuating that the reviewer didn't do his job in the first place. Yeah, yeah, I know 1up.com's Shane Bettenhausen famously did NOT finish The Legend of Zelda: Twilight Princess before reviewing it, but for the most part you're going to get someone who does his due diligence as long as he's not writing for, oh, Maxim. (And for the record, Shane was reportedly at or near the final boss before he drafted his review.)

There are certain instances, however, where the woman's statement does make sense -- though it's not in the context of her in-store exchange with the guy. Alex's Guitar Hero III story serves not only as an anecdote of how thorough reviewers are but also a launchpad for supporting the lady's statement. With games based heavily on deep online activity that evolves as the community grows, it's understandably impossible to always be right about how good a game is without waiting for some time. Take 1up.com's Joe Rybicki's review of SOCOM: Confrontation for the Playstation 3, for example. SOCOM: Confrontation launched with horrendous glitches. Joe gave the game the appropriate score, with a vow to re-evaluate it if and when a patch became available. That he did. (And let me be clear, for those of you pointing to my previous paragraph: A "glitch" is not tantamount to a design flaw or poor planning.)

Games Get Old: "That's because it just came out."

Ultimately, for the games that aren't online heavy and that remain as one static product throughout its life cycle, it is true that a year from now Game X might not be as enticing as it once was. It's easy to look at a review for a game and wonder how in the world that game got praised so highly when its good traits are found everywhere.

Well, duh. Games get old. Some games do age well, sure, but of course you're going to find the development houses that "do what Game X did, but better!!!11". That said, that's why games are reviewed as close to launch as practically possible, when most of the sales usually happen. If all you did was wait until later for games to come out, and then making your own evaluation based on the fact that it's old hat, then when will you ever buy and enjoy a game? It's similar (though not the same, I realize) to those with PC upgrade dilemmas. If all you're doing is waiting for the next big thing at the perfect price, you're never going to get your PC built becaues something is going to come along that is flat-out better. It's not that games will continue to get better at a linear rate year after year, but all I'm trying to say here is that the mentality asserting that a game's reviews are great because it just came out is counter-productive and flawed. When a game releases, you most likely want to know where it stands right here and right now. That's why you're buying the game now. I'd hope that, if you decided to wait on a game, you'd go into that store a year later fully understanding that the game has been out for a while and that new, shinier toys are coming along the horizon. Whether or not your experience is tainted by the knowledge that the game is old is a personal thing -- it's not in the hands of a reviewer to predict how said game will turn out months or years from now.

Look. Since games have existed, and at least for the forseeable future (diminishing returns, remember), this medium is largely technology-based. The things you can do within your imaginary world -- hell, the complexity of the imaginary world you can achieve -- are largely dictated by how powerful your box is, and how adept you are at working with the box. As time goes on, tricks will be found within a hardware generation, and new generations of hardware will offer more opportunities. For every game that remains as fun today as it was ten years ago, there's a game that would only benefit from new hardware. Are you really going to try to invalidate a reviewer's evaluation of a game just because the reviewer chose not to take the future into account? What's the use of reviews then? The use, again, is in the here and now: As one of the hundreds of thousands of gaming fans who've lined up for some game so that you could have it on launch day or soon thereafter, you want to know if it's worth waiting on that line -- not whether or not you should be waiting for the next console's powerhouse hardware to rear its head.

Sidenote: It should be mentioned that this is why the subject of re-reviews is a sticky one. That Joe re-reviewed SOCOM: Confrontation is not an issue to me as a consumer; it's good to know that if I buy the game today, it works well. That said, there's the tricky task of setting a critical standard with reviewing games that could theoretically be patched. It might not make practical sense to continually go back to a game and keep on reviewing it with every new patch. Equate that to the user experience, where the user has to sit through a multitude of patch downloads (unless the developer is kind enough to offer a cumulative download) just to get his game working right... or worse, where the user can't get the updates because he's a single-player kind of guy and is stuck without a network connection. In any case, Gamespot offers editor's notes and updates for any games that receive an update that addresses a previously criticized issue.

So what do you think? Are you wary of a game's positive reviews because "it just came out?" Why or why not?

#1 Posted by MrCHUP0N (246 posts) -

Are a game's positive reviews rendered moot because they were written right when the game in question was released? If you think this is a ridiculous sentiment, hold on just a bit while I recount my experience a few days ago. I was in my local Gamestop on Sunday, picking up a long-overdue copy of Advance Wars: Days of Ruin. Behind me in line was a couple waiting to pay for Fallout 3 for the Xbox 360. The woman remarked, "Wow, so I heard this game got really good reviews." The man replied, almost without hesitation, "Yeah, but that's because it just came out."

The first thought to pop into my head was, "0_o" -- but I told myself to listen on in case the guy had a follow-up. "Everybody gets caught up in the excitement," he explained to the woman, "so everyone's really looking forward to playing this new thing and they're happy to finally get their hands on it." She replied, "Ah. So it probably makes more sense to wait and buy it later when people start to see what might be wrong with it."

As you can probably imagine, "o_0" was still lingering in my head. Now, it's not as if what they were saying was completely devoid of merit, but at least on the surface it's plain to see that there were a few incorrect assumptions being made about what (good) reviewers do and how they come up with the evaluations that they do. What could drive the belief that a highly positive review is based on its proximity to the game's release?

The Hype Factor: "Everybody gets caught up in the excitement"

It certainly feels good to get your hands on a highly-anticipated game for the first time, and for some critics, the early access makes it even better. I argue, however, that professional critics don't get to the position that they're in by falling heads-over-heels for every new hot title. Part of the job is being able to look at every product from as objective an angle as possible. The critic is telling you whether or not to spend not only hours of your life but also SIXTY DOLLARS on the game in question -- and quite frankly, I would personally rather debate with someone accusing me of low-balling a game than feel guilty about having someone upset at spending time and money on a game he ultimately didn't enjoy. That's not to say I approach games with the intention of being overly critical, but rather I approach with caution.

Taking it one step further...

Covering All Your Bases: "It probably makes more sense to wait [for] when people start to see what might be wrong with [the game]."

This is a difficult one to pin down, but first let's address what's potentially wrong with this thinking. If we're being cynical and bitter, it's easy to say that this statement is made under the assumption that game reviewers don't thoroughly play the game in question. After all, what's the logic behind saying that one won't know what's wrong with a game until it's been out for a few months? (I can hear, "Yeah, but..." coming -- I'll get to you, don't worry.)

A reviewer generally plays a game from start to finish, or at the very least to an extent where an educated opinion can be made (to me, that means "finished"). A role-playing game isn't abandoned after only half of the adventure has been complete. A fighting game isn't dropped after playing in a couple of scattered bouts. A reviewer should be as complete as possible with the game in question, not only to make sure that as many issues as possible have been uncovered but also to make sure that any residual excitement from hype -- for those prone to such a thing -- has died down. In my experience, this is surely the case.

It's storytime again, and I promise you it'll be short. Let's jump back to erstwhile Associate Editor Alex Navarro, now at Harmonix. When Guitar Hero III launched, the man posted his review after the product launch all because he wanted to give you an accurate and thorough representation of how online worked against real competition. Hell -- the review code's online play was buggy, so he lugged his butt to Best Buy, picked up a retail copy of Guitar Hero III for each console, and hauled the plastic motherlode back to his reviewing dungeon over the weekend just to make sure he could get in some quality time.

So, yes -- good reviewers have been and will continue to be thorough and complete in assessing a game. No matter how long a game has been in the market for, poor design decisions, planning and ideas won't go away, or suddenly appear out of thin air, or get any worse or better (put your hand down -- I'm getting there). For the most part, to say that a game would review better with time is insinuating that the reviewer didn't do his job in the first place. Yeah, yeah, I know 1up.com's Shane Bettenhausen famously did NOT finish The Legend of Zelda: Twilight Princess before reviewing it, but for the most part you're going to get someone who does his due diligence as long as he's not writing for, oh, Maxim. (And for the record, Shane was reportedly at or near the final boss before he drafted his review.)

There are certain instances, however, where the woman's statement does make sense -- though it's not in the context of her in-store exchange with the guy. Alex's Guitar Hero III story serves not only as an anecdote of how thorough reviewers are but also a launchpad for supporting the lady's statement. With games based heavily on deep online activity that evolves as the community grows, it's understandably impossible to always be right about how good a game is without waiting for some time. Take 1up.com's Joe Rybicki's review of SOCOM: Confrontation for the Playstation 3, for example. SOCOM: Confrontation launched with horrendous glitches. Joe gave the game the appropriate score, with a vow to re-evaluate it if and when a patch became available. That he did. (And let me be clear, for those of you pointing to my previous paragraph: A "glitch" is not tantamount to a design flaw or poor planning.)

Games Get Old: "That's because it just came out."

Ultimately, for the games that aren't online heavy and that remain as one static product throughout its life cycle, it is true that a year from now Game X might not be as enticing as it once was. It's easy to look at a review for a game and wonder how in the world that game got praised so highly when its good traits are found everywhere.

Well, duh. Games get old. Some games do age well, sure, but of course you're going to find the development houses that "do what Game X did, but better!!!11". That said, that's why games are reviewed as close to launch as practically possible, when most of the sales usually happen. If all you did was wait until later for games to come out, and then making your own evaluation based on the fact that it's old hat, then when will you ever buy and enjoy a game? It's similar (though not the same, I realize) to those with PC upgrade dilemmas. If all you're doing is waiting for the next big thing at the perfect price, you're never going to get your PC built becaues something is going to come along that is flat-out better. It's not that games will continue to get better at a linear rate year after year, but all I'm trying to say here is that the mentality asserting that a game's reviews are great because it just came out is counter-productive and flawed. When a game releases, you most likely want to know where it stands right here and right now. That's why you're buying the game now. I'd hope that, if you decided to wait on a game, you'd go into that store a year later fully understanding that the game has been out for a while and that new, shinier toys are coming along the horizon. Whether or not your experience is tainted by the knowledge that the game is old is a personal thing -- it's not in the hands of a reviewer to predict how said game will turn out months or years from now.

Look. Since games have existed, and at least for the forseeable future (diminishing returns, remember), this medium is largely technology-based. The things you can do within your imaginary world -- hell, the complexity of the imaginary world you can achieve -- are largely dictated by how powerful your box is, and how adept you are at working with the box. As time goes on, tricks will be found within a hardware generation, and new generations of hardware will offer more opportunities. For every game that remains as fun today as it was ten years ago, there's a game that would only benefit from new hardware. Are you really going to try to invalidate a reviewer's evaluation of a game just because the reviewer chose not to take the future into account? What's the use of reviews then? The use, again, is in the here and now: As one of the hundreds of thousands of gaming fans who've lined up for some game so that you could have it on launch day or soon thereafter, you want to know if it's worth waiting on that line -- not whether or not you should be waiting for the next console's powerhouse hardware to rear its head.

Sidenote: It should be mentioned that this is why the subject of re-reviews is a sticky one. That Joe re-reviewed SOCOM: Confrontation is not an issue to me as a consumer; it's good to know that if I buy the game today, it works well. That said, there's the tricky task of setting a critical standard with reviewing games that could theoretically be patched. It might not make practical sense to continually go back to a game and keep on reviewing it with every new patch. Equate that to the user experience, where the user has to sit through a multitude of patch downloads (unless the developer is kind enough to offer a cumulative download) just to get his game working right... or worse, where the user can't get the updates because he's a single-player kind of guy and is stuck without a network connection. In any case, Gamespot offers editor's notes and updates for any games that receive an update that addresses a previously criticized issue.

So what do you think? Are you wary of a game's positive reviews because "it just came out?" Why or why not?

#2 Posted by MichaelScott (193 posts) -

I don't think "it's getting positive feedback because it just came out" is a fair statement, at all. Rather, I think if people are excited for the game--especially game reviewers--they will be extremely disappointing if the game fails to meet their expectations and they will rate it much lower because of that. Reviewers get the game early so they can have their reviews out before the game is released, so the consumer can make a decision sooner. That's the business. Does anyone think Super Mario Galaxy got raving reviews because "people were excited for it?" This whole argument suggest that video game journalists have no standards and don't thoroughly review games with an objective view. While this is probably true for some outlets, certainly this is not true for all video game journalists.

If everyone reviewed games on hype, some games would get much higher review scores(i.e. Assassin's creed. Although it got pretty good scores regardless, many were disappointed with the game.).

About the whole "making sure the game plays well with other people online" thing, well, I really think that's irrelevant. As long as the online works, all you need to do as a writer is say that it works, and it's just as good or better than the core game(or worse.). Just because playing it with random people is such a variable. You could play with some complete jerks and they will just mess up on purpose. On the other hand, you could play with some really cool guys and have a good time. The server condition is another variable. You can't judge a game based on variable like that, so you can't add or sub tract from the score unless the online is totally broken and doesn't work at all. You should be able to test the online before a game comes out with a few people. (other reviewers,people in the office, etc.)

I'm not wary of a games positive reviews being positive because they just came out. If you read reviews like that, it will be very easy to tell if that's the case or not.

#3 Posted by MrCHUP0N (246 posts) -

The thing about making sure the game plays well online is absolutely relevant and cannot be ignored. However, I wasn't entirely clear so I'll try to be more specific. What I'm referring to isn't whether or not you'll encounter asshats. Rather, a reviewer should do the due diligence to see -- for example, in a shooter -- whether or not maps and weapons are balanced; whether there's any notable evolution in gameplay as the game continues to be played (Call of Duty 4 is a prime example); whether or not the experience as a whole is conducive to a whole bunch of humans playing against and with each other. These are things that become more clear with at least some time spent with the real world.

In Alex Navarro's case, he had to get an accurate representation of whether or not the average Guitar Hero III online experience was playable, and he couldn't do that with buggy review code. Yes, server capabilities do vary. But yes, you can assess over a period of time whether or not the average experience is competent. Could the issues be due to bugs that did not pass QA? Could the issues be due to poor design on the developer's part? What happens if you test a game within the office -- where each box is hooked up to a high-speed LAN -- and experience smooth gameplay, and then when average Joes buy the game, there's widespread lag? The idea is not to "blame the game" but to simply give the audience what it needs to know: Will it play well in general? You can't tell everyone that they're going to have a great experience or poor lag, but you can at least give them a ballpark.

All told, you can't just say "the online mode works" and leave it that. That's not doing your job, period.

#4 Posted by Tiago (92 posts) -

GTA IV

'nuff said.

#5 Edited by MichaelScott (193 posts) -

I misunderstood you. My mistake.

I'll concede your points about the online factor. Past that I have nothing more to add right now. Fine blog post, sir. :)

#6 Posted by Dovey (161 posts) -

Hey Chu, very nice blog entry raising up an interesting topic.

There isn't really any simple answer for me to the question that you have raised but If I was to pick one side of the coin from the other, it would be that I'm not wary of positive reviews for games just released.

I myself would have probably made up my mind on a purchase before the review and it would take an extreme case to change my mind (Alone in the Dark for example).

But reviews are wriiten to inform not to help conform people to a purchase.