Polygon changes review score...again

Yesterday, Polygon posted their review of SimCity and gave it a score of 9.5. The game was reviewed using the pre-release servers that EA had set up for advance copies. A lot of outlets chose to wait for the live servers to launch to publish their reviews but Polygon and some notable others didn't. Penny Arcade's Ben Kuchera wrote an article saying that readers shouldn't trust any SimCity review that goes up before knowing what the live servers would be.

Lo and behold, the live servers went up and the result has been mostly a disaster. Some people have been unable to get into the game at all, while others can only get in sporadically. So, just hours after its original review was published, Polygon revised the review and dropped SimCity's score from 9.5 to 8.

There's a lot of ways to view this. On one hand, Polygon should probably be commended by revising the review and trying to provide the most accurate view possible. On the other hand, one could argue that this could have been avoided by simply waiting an extra day to publish the review. What does Polygon say to those people who bought the game thinking it was a 9.5, only to find out a few hours later that it's actually an 8? It's a interesting precedent being set here and one that I'm not all that sure I'm behind. What say you, Giant Bomb?

UPDATE: After continuing server instability over the last few days, Polygon has once again dropped the review score, now down to a 4.

Kind of digs them a hole. If the servers get fixed tomorrow, does the score go back to a 9.5? or is this score permanent? Doesn't seem fair to nerf the score when something is broken and not replace it when they fix it.

Huh, that was unexpected but a number is still just a number, it's all in the text of the review.

That's.... Wrong.

Credibilitygon

Jumpedthegon

Ummm.....

We're so spoiled in that we get to watch extra videos like Jar Time with Jeff that help us get to know the thought process behind how they review their games and the person doing the review. Which in a way can help us understand how they come to a score and what that person looks for in a game.

With sites like Polygon you don't get any of that so the guy doing the review might as well be a mystical robot man who plays both sides of the table. So to me, none of what they're doing is really going to help their image in any way. The only thing I can see to help gain trust is to start showing videos of them like Giant Bomb.

Lol, Polygon.

I'm wondering this too. If the servers are fixed in a couple days (which I assume they will be), does Polygon go back and say "oops, nevermind! we were right the first time!"? It just eventually seems like an exercise in futility. Who are you serving at that point?

Hm that is certainly odd. It's not like the game somehow got worse other than the usual launch day issues of a game that requires online use at all times.

They did say beforehand that they would change the score based on possible release issues because they reviewed it on review servers.

Exactly as you say.

On the one hand, good on them for changing the score if they felt like it ended up not reflecting the final product. Something Jeff has been banging on about for a while now is that reviews are static, you write them and they're done, but if a game changes dramatically since the review goes live, that review is useless. Maybe this could set a precedent and more reviewers revise their reviews post-launch.

But, changing the score, especially at the last minute like that, could have caused someone to buy a game they might not have before, although in this case if someone feels cheated for buying an "8/10" game instead of a "9.5/10" game, they should get their head examined.

Good on them

Seems hasty.

Plus this style of "editing the review" seems like a copout quite honestly.

Makes no sense to me. Although I hate to say it, this is a case where you should look to ESRB. They rate the game for what it is, and any part of the game that might change from day to day is mentioned as unrated. It should be clearly mentioned in the review that the game requires always online etc and that the review does not reflect the quality of that service.

For sure. And changing review scores has been part of their policy since day one. So, it's not like it's just something they made up for this special case. But if your anticipating these kind of issues, why publish the review in the first place? The 9.5 is representative of the game on pre-release servers, which is useful to no one.

I'm not offended by it and I don't get why a consumer would be. But I think it shows that Polygon is playing the IGN-game of being the first or nearly first up with a review and are now paying for it. I'm not sure Polygon should take part in that game because it doesn't do much to bolster their journalistic integrity.

And what if the server issues clear up tomorrow, or the next day? Is the game still an 8 and not a 9.5?

This whole thing is stupid.

That's a little bit of bull. The servers allow people to access the game, sure, but they're supposed to be reviewing the game itself, not the ease of access to it.

Dammit! I thought it was Arthur Gies. ~ Still interesting.

This doesn't work for me. Read over their change summary/justification:

As many worried, today's launch of SimCity has brought a number of server woes and instability with it. Some players are unable to connect to EA's servers to download the game. Others are unable to sign into SimCity's always-online service to start a game. Others are suffering from disconnections while in-game, which often results in lost progress and bizarre glitches. Our own reviewer, Russ Pitts, has suffered disconnected sessions this afternoon that resulted in lost progress, corrupted avenue placement, and twin monster attacks.

After speaking with Russ and Polygon Managing Editor Justin McElroy, we are in agreement that the current state of SimCity merits an update to the game's score on Polygon, per our reviews policy. While not every player is experiencing these problems, members of our staff, other members of the press, and an anecdotally large portion of our readership are having moderate to severe difficulty playing the game. This likely-temporary scenario nonetheless affects our recommendation of SimCity, and we advise caution for the time being before diving headfirst into the game. - Arthur Gies, Polygon Reviews Editor

I don't understand how dropping the score a point and a half accomplishes their goal of advising caution. It's still an 8. If they really wanted to advise caution, why not drop the score to, I don't know, something actually bad. Something that would legitimately advise some caution. As it stands, the score at face value only says the game is slightly less good, but still good. Is an unplayable game good? I don't know, man.

It's their site to run, but if it were me, maybe I'd have just left the score as is, but added an update at the top of the review warning about the complications.

HEY GUISE MAYBEE NOT PREORDAR EA GAIMS NEMORE OR BLINDLY TRUSST RVIEW SCOARS

If it wasn't a necessary part of the game, than sure, it wouldn't matter. But when some people can't connect to the game, or lose progress because the servers a fucked up, that's entirely on Sim City because there is no game without the servers.

On the other hand, one could argue that this could have been avoided by simply waiting an extra day to publish the review. What does Polygon say to those people to bought the game thinking it was a 9.5, only to find out a few hours later that it's actually an 8?

This, what is the worth of a review that changes significantly in such as short period of time? When the server issues are worked out does the score return to a 9.5? It seems really reactionary. More seriously though why review a game before you know whether it can work in true conditions, especially when you consider how predictable server issues are when always online games are first released, there was a reason many sites didn't off day 1 reviews.

In a normal scenario with no server issues, they viewed it as a 9.5 but at launch it's not at that level of quality. But the server issues will most likely subside for the most part so I can definitely see the reason for publishing it. I'm more curious if they will revise the score again once this issue is resolved.

@LikeaSsur: In an always-online game...this is an absolutely integral part of the experience and should ABSOLUTELY be addressed in the review.

So here's the thing, I (now) get that they have this fluid review score policy thing going on (with a history of any changes), which is fair enough I suppose, but in that case why would you decide to temporarily revise it to 8/10? 'You might not even be able to play the game at the moment! 8/10!'

They should temporarily revise the review score to Fucked/10 until such time as the game works reliably.

While I understand their thinking behind this practice, it is just downright goofy (and kind of stupid) in execution.

The servers will most likely be working fine in a day or two. If they are, does the score go back up to 9.5? If so, what good did it really do at 8? What's the real sizeable difference between a 9.5 and an 8 in terms of people's buying decisions? If a server problem is making the game literally unplayable than shouldn't the game's score be something like 0 or N/A since there are some people who can't actually play it?

It just seems like Polygon wanted the day one traffic so they worked out some little pre-release deal with EA to play it, than they wanted to how off their cool, innovative new score altering system when the servers buckled a bit (which we all knew they would) and in the end, it won't matter at all. It just seems goofy.

Additionally, I think reviews should be about one person's experience with the game, not some idyllic reflection of the objective score balanced between all person's experience because that is not possible and not all that helpful to attempt.

You really hut the nail on the head.

The same thing has to be wondered about Diablo 3, it has a borked release. Alot of reviewers reviewed as is day 1, while with all the launch problems gone, its a different score.

However with Polygon doing it in the reverse, makes go me go...what changes scores with them then? Shouldn't they change scores based of age as-well? Sure many sites reviewed Shadowman on ps1 as a 4 or such...but today it would be a 0!

I think reviewer scores should be treated like the games themselves. The review should be as fine as its ever going to be day one, same with games. However games now freely ship broken version and throw day 1 patches to fix them ontop. This is just a day 1 patch for a review. Except to turn on my own argument...how can a reveiw be accurate with game doing this practise!? Simple....DONT SHIP BROKEN GAMES! I know the sim city issues not exactly the same, however the list they get for Cod and MOH and the likes before being allowed to review it.....

Reviews are about trust, this does not signifie trust even if it was for a better reason than "OH NOES rival sites make us look crazy! Lets change the score to seem sane!" (E.G GTA 4 syndrome.)

Sounds like a panicky move to me, seconding the curiosity about them revising it again.

So stupid. I don't see how the game's shitty launch changes the contents of the actual game.

Ought to be a disclaimer or noted in the review that there's launch troubles, but coming back on your review saying "awww well they handled the launch like ass so 1.5 points off". In a week will they bump it up to 9.5 again?

Ridiculous.

I don't like it. It should have been 8 to start with (6 in my opinion). It should have been 8 to begin with because it has online restrictions. Everyone knew this would occur on release day, if they wanted to really have some balls they could have given a predicted score of 5 beforehand, based on expected release day problems.

I'm really enjoying the twitter beef about this issue so far.

I don't understand how server issues effect the actual quality of gameplay. I mean yeah I get it, if the servers are bad, then you can't play for a while, but that doesn't effect the actual GAME itself. The review should be about the quality of the game in my opinion, not the status of the launch day servers. If this was a persistent issue, maybe, but dopping a game down from 9.5 to 8 because of server issues seems weird to me.

This is too soon, too reactionary. Release day server issues don't warrant a bump in the score. Understandable if a patch ruined a game for the reviewer down the line eg DayZ, but SimCity's only been out for a day.

Changing the review score because of launch day issues is complete bullshit. In a month, how relevant will that be? If the game is a 9.5 when it works, the game is a fucking 9.5. If, in a month or two when things should be working the game is still fucked, sure, maybe your pre-release score was wrong and you should revisit it. But that is the dumbest thing I've heard.

@zudthespud said:

Many of their complaints seem to simply revolve around the general bugginess of the experience. That's always the problem with rating a game based on initial impressions (which, judging by when this review code went out, 'initial impressions' is exactly what they are).

You're comparing the theoretical experience with the actual experience. Regardless of what you define as what reviewers are 'reviewing,' the end-goal is to review the customer experience. The issues cited in this revision ("Some players are unable to connect to EA's servers to download the game. Others are unable to sign into SimCity's always-online service to start a game. Others are suffering from disconnections while in-game, which often results in lost progress and bizarre glitches. Our own reviewer, Russ Pitts, has suffered disconnected sessions this afternoon that resulted in lost progress, corrupted avenue placement, and twin monster attacks.") are all direct examples of a really poor customer experience. It's like complaining about a really buggy experience: theoretically, they aren't part of the gameplay experience, but they can certainly affect someone's enjoyment of a game regardless of their anecdotal and unpredictable nature.

Should reviewers also ignore the typical 'buggy MMO launch' because it's only a temporary thing? I, personally, don't think so. By doing so, you're separating the abstract, essential 'design' of a game from what it manifests itself as in reality--a reality that people spend money on.

"8" is still a good score, right? I mean we haven't devolved so much that even 8's are now considered mediocre, have we?

well- it does make me question their priorities a bit...clearly they thought it was of utmost importance to have a timely review, rather than a realistic one. but they aren't alone in that.

it doesn't really bother me, as i don't trust a single source for reviews, AND they amended the review within a matter of hours. now if they had the early review exclusive for a week with 9.5, and change it to an 8 on launch...that's FAR more repulsive.

So... the game being mostly unplayable for a few days only takes off 1.5?? What? Shouldn't it be a 0/10 or better yet, what was posted earlier in this thread, a Fucked/10.

It would be better if there was no score at all and the review just started with "GAME IS BORKED! IT'S VERY LIKELY YOU CAN'T PLAY IT YET!" and then they could remove it when it was fixed. But, yeah, no score probably means less traffic or something.

For real, what does changing the score accomplish?

Pandering.

Oh hey, that score bump is really going to mean a lot to someone coming to play this game past release week. As useless as any Diablo 3 review that just said "Error 37 derp!"

I can continue to think Polygon is dumb as hell. Fantastic!

An 8/10 seems to translate to a 3/5 these days.

Seems kinda crazy being as the servers could suddenly start to work again in an hour or two. It's not as if the game will never be playable after its released after all. And as many people have already pointed out waiting to publish a review until at least the day the game comes out makes sense, let the issues with the live servers that could occur at least have a mention in the review.

It's bizarre. As others have noted, a drop from 9.5 to 8 is relatively trivial considering the stated reason is the nigh-unplayability many users are experiencing. Surely it would have made more sense to withdraw any score until the issues are solved?

On another note, the review itself is surprisingly positive considering what I've been hearing from Jeff, Penny Arcade Report, Ars Technica and others about the restrictions and oddities and the bugs - dear god, the bugs. Everyone's different I guess but I dunno. Polygon's weird.

This is the big problem with the method of reviews becoming living documents. Really though, they should have never posted a review for a server dependent game without basing it on real world servers, instead of preview builds

Almost everything about Polygon bugs me. This just adds one more log to the fire.

It doesn't matter what review scores a game receives; We've seen this happen in the past enough times that everyone should have been aware that it would happen with Sim City. Remember the Diablo 3 release? Not to mention EA is not the most reliable, or customer friendly publisher. People need to learn to be patient; wait a week or two after release before you slap down $60. If you purchase a game on release day based on one or two reviews, it's your own fault if you fall prey to issues such as this. I was really looking forward to this game, but after reading some reviews, and seeing the nonsense with the servers, I am glad I exercised some patience.

I dunno man. I'm not a huge fan of the site (though it is growing on me) but I'd say most of the staff there is easily as accessible as the Giant Bomb's.

Did you know they actually post in their forums? And not just to make snarky comments at people who disagree with their opinion.

On topic: I respect that they changed the score more than I would respect them leaving for the sake of saving face.

The shitstorm this created on twitter is quite hilarious

I *get* the idea of updating scores. If a game is a buggy mess and is later patched to pristine polish, I could probably get behind bumping the score (if that was a negative to it). But this is just taking that to the logical extreme.