Review Scores: Pre-release vs. Post-release

Just something I found interesting. The average review score on Metacritic for SimCity is dramatically different based on whether the review was posted before or after the review date. Before the game released, the average was 89. However, the scores posted after the release averaged 70 (as of today, March 8th). With this, I'm wondering if games which require an always-online connection should even be reviewed before they are released and the servers are truly tested. What do you guys think?

EDIT: Fixed my funky math on the scores.

In general I think it's long been a mistake that games receive static scores. Since the 80s the truth of non-console games has been that they are not static. They can be patched and changed, and, as we see more and more these days, their servers can crash and burn. Games can get better, and worse. Why shouldn't the review reflect that? As long as version history is expressed this is clearly the way to go.

Now that consoles are open to Expansion Packs and patching like the PC it's not hard to imagine that many review sites will start to incorporate a Post-Review Lean, where they can adjust score for network stability and other updates that may have landed (or features that were removed). I also think DLC should figure here but that's another topic.

One problem is the manpower required to be consistent about such a system. Should outlets re-review all the games they review? Or is it OK to focus on the popular ones? I noticed on Polygon (not dissing them but it's relevant) that they've changed the score for SimCity many times. However, a patch was also recently released for Aliens Colonial Marines. Should this be reflected in the review? How serious does the patch or networking issue or feature update/decrease need to be to affect a change in the score?

Good points. Perhaps SimCity should be re-reviewed when (if) the online issues are fixed. I didn't even know that Polygon had changed their review (twice), but I think it's great that they kept the history of their reviews. But what I'm wondering is if reviews should even be done in the first place for an unreleased game that relies on the company's servers. The scores before the release I believe accurately reflect the game's potential, but the scores after the release accurately reflect the reality, at least today's reality. But if all of this is fixed next week, then the scores will once again be inaccurate.

Reviews are really weird the more and more I think about it.

@alfawolf04: I think the best answer to this is that anyone who reviewed it before its release did it wrong. Many publications waited for live servers to publish their reviews, knowing that something like this might happen. In my mind, only the lesser publications/sites would do something like that.

If a game is using any sort of always-on scheme it should never under any circumstances be reviewed before launch and any reviews published before launch of said games should be ignored until the game is out and the situation can be fully assessed.

As for static vs dynamic scores consider that gaming sites can barely keep up with everything being released that dedicating any resources to covering one game for extended periods of time is just wishful thinking. At best they do it with a few big games and what comes out of it is this incredible lack of consistency in standards and ethics on how the site review games.

I think it only makes sense to not review a game that is using online only drm such as SimCity. Because the online functionality now is game mechanic, not an extra feature like a mp mode, it needs to be considered when reviewing a game. While I do like that Polygon has this flexibility in their reviews, obviously they wouldn't have had to do this if they'd just waited a day or two to make sure the servers would be stable enough for people to play the game.

While I like the idea of continually updating or revising the review of a game, its wishful thinking on my part that any site would have the manpower to constantly update the review to reflect the state of the game while doing new reviews for new games at the same time.

Reviewing a game before it is out when the reviewer knows the gameplay experience could be subject to changes post release is wrong. I don't care if you change the score or pull the review, stop chasing clicks and start doing your fucking jobs properly. Alex did the right thing by waiting until after release to see if the servers could handle the rush of new players, they couldn't, his score and review reflects that. Its called professionalism and it'd be nice to see it a lot more often in this industry.

Games that require constant internet connections should be held off from reviews till at least launch or the day after so the servers for the public can be examined and the reviewer can judge the game accordingly. If Sim City has showed us anything it is that games relying on servers are very static and anything can go wrong on launch day when you get the mass public all trying to play and everything just breaks, review sites like Polygon don't need egg on their face when they have to change their review score because they tried to be the first out the door with their review instead of having the patience to wait.

Sites shouldn't write reviews for always-online games until at least a week after release. Daily impressions up until that point would be more useful.

so here's my take:

i think pre-orders are the fundamental problem. if no one preordered, there wouldn't be a need for reviews prior to the release (since most gamers treat reviews as de facto buying 'advice.') but the practice of preorders isn't likely to stop anytime soon- so publications have a choice...either A) take the hardline stance of only reviewing final code running on consumer-end infrastructure (at the expense of timeliness and leaving those who preordered high-and-dry) or B) reviewing games as they receive builds, with visible disclosures of their reviewing conditions and perhaps revisiting these builds at a later date.

to be honest, neither scenario seems ideal. but i generally don't pre-order unless it's a sure bet (heart of the swarm and me3 were the last 2 games i preordered, and they're fairly well known quantities), and won't purchase games until i've read multiple reviews. so either option works for me, really, as i'm almost never a day 1 shopper.

Reviewing a game as a good and reviewing a game as a service are two different beasts. One one hand you have a game that is a finished product on release and may receive some DLC here and there, but for all intents and purposes that game is what you expect, even if it's 10 years from now. Games as services, however, are constantly changing. Those old reviews of games like MapleStory might as well be useless, as that game and others have changed completely from their respective releases until the present day. Can we honestly expect every reviewer to change their reviews and the scores every time a new update comes out or something horribly wrong affects the experience? How should a dynamic review work to inform the people of a game's current state? Obviously, we haven't figured out the best way to handle them. Games like SimCity and reviews of it like Polygon's are challenging and changing our views on what reviews are supposed to do.

Times are changing.

@gruebacca said:

Maybe they should review game updates and patches individually that stand alongside the original review.

Reviews and their scores are outdated. You really can never know if the person behind it is being paid to give it above a certain score...and I've played plenty of games my favorite reviewers have raved about but I've hated.

And for this post-release change of score...we should give Baldur's Gate a 10.1 since user-made patches allowed it to be played in BG2's engine. But seriously...a bigger change needs to happen.

I think the whole point of Polygon's dynamic scoring system is so that games like SimCity can be reviewed pre-release, and then be updated if they don't deliver on the promise made by what they played pre-release. And just so it's clear, one day when SimCity gets its shit together, they can bump the score back up.