Guns of Navarro

Posted by Alex (2368 posts) -

SimCity has been out for, as of this piece's publishing, five days. For those five days, players have been unable to play SimCity, save but for brief, taunting periods of functionality. The online servers remain clogged and/or offline, which is a problem given the game's insistence that players be online in order to play at all. Maxis has come out and apologized directly for all of this, while making claims to improved, but still problematic server functionality, and proffering a token of peace in the form of a free (as-yet-undetermined) EA game. But while those reports of improvement are great to hear, people still can't play across the board, and even working servers are still running into issues, despite the shut-off of some in-game services. As it stands, a game that people paid $60 for barely functions, if at all, and people are understandably upset.

SimCity should have been an easy game to fall in love with, but between its design quirks and abysmal online service at launch, it's been nothing short of a disaster so far.

SimCity has other problems with its design, many of which I laid out in my review, but the talk of the past week has been the servers, as well as the reviews that have or have not appeared in the wake of the game's shoddy launch. It's not hard to understand why. For whatever design flaws SimCity might have, the simple fact of the matter is that people, including many reviewers who chose to wait until post-launch to test the game, cannot play it. And those that did choose to review it prior to launch are suddenly catching no small amount of shit from the SimCity community for not taking the game's server performance into account prior to reviewing.

In all my years critiquing games, this is maybe the weirdest launch week I've ever found myself a part of. I've written and edited my way through numerous MMO launches, online multiplayer games, and, of course, last year's similar debacle involving the always-online Diablo III. When that game came around, I remember there was some trepidation regarding Blizzard's ability to handle the surge of players come launch, especially given the rocky start of Blizzard's bread-and-butter, World of Warcraft, back in its first year. Still, while Diablo III still ran into server issues for some time following release, it did manage to iron out its most egregious connection wrinkles relatively quickly compared to SimCity, which EA is still, several days later, trying to salve with additional servers.

EA's line on all of this is that they didn't expect the volume of players they received, nor did they expect them to play for the long stretches they've apparently been playing (provided they're actually able to get into the game in the first place). As someone who has gone through several sometimes rocky website launches (including one fairly recent one some of you may recall), it's not that I don't have some sympathy for the developers who are forced to try and fix this while everyone is shouting at them from every angle. You can't test for everything, and weird things will always pop up in the live environment that you can't account for. That said, I would echo the sentiment others have expressed that EA's insistence on hour-limited gaming sessions during SimCity's beta probably didn't do them any favors.

Regardless of what went wrong, the fact of the matter is that we're here, now, with a fairly broken game that has nonetheless garnered relatively high reviews from those who were able to play it, as well as fairly negative ones from those who have run into various problems trying to play it. Out of that strange dichotomy of reviews has evolved a salient pair of questions. Namely, if games are as much about their services as they are the core gameplay--as Joystiq's Alexander Sliwinski smartly poses in his editorial on the subject--should we be reviewing them before they launch? And should the scores be adjusted to reflect the ever-evolving state of such games?

Let's tackle that first part. When I started at GameSpot back in 2003, even then we had a policy that more or less dictated that all online games should be tested in an environment that satisfactorily emulates the retail experience. Early in my tenure, that policy made a lot of sense, as online multiplayer games often had a tendency to feature server problems, bugs, and whatever else that you might not see just playing against other journalists on pre-release servers. In recent years, this has become somewhat less necessary, as live servers have gone up earlier and earlier prior to games' launches, and retail copies have become more readily available to press prior to release. However, for a game like, say, an MMO, nobody in their right mind would review it without spending ample time on retail servers. Granted, that's usually because with games like that, you literally can't play it until it's live for everybody. But in the rare case that you would be granted some kind of early access, it still wouldn't matter, since so much of the game's content pertains to the online experience.

SimCity operates in a weird middle-ground between those two examples. In SimCity, you can absolutely see the content of the game even if only a few players are around and servers are up. EA provided multiple critics (including myself) early codes that ran on servers specifically set up by the devs for review processes. It was made abundantly clear in the various communiques between press and PR that these would not necessarily be indicative of the live experience, but that we could use these to get started. That was exactly the attitude I took when taking to the game last weekend. I wanted to see everything I could, as I was under the impression that some kind of server catastrophe could happen. Though I wouldn't say I expected anything quite like this.

Ask most industry prognosticators whether or not they saw the disastrous launch of EA's online-always SimCity coming, and they'll nearly universally answer yes. That might sound odd, considering there weren't alarm bells being sounded on a regular basis prior to the game's launch. If you go back and read most pre-release coverage of SimCity, you might see various writers making mention of the game's always-connected infrastructure, perhaps even noting some reservations with the idea. But doom and gloom was really nowhere of any obviousness to be found. I think there is perhaps a reasonable reluctance among those in our trade to appear too negative without having solid experiences to back that up. At the same time, most of us are dyed-in-the-wool cynics, so when someone says to us, "Yeah, we're gonna be online all the time," our reaction is usually to just smile, nod, and hope for the best, even though deep down, we have an inkling that this is all going to go horribly wrong.

Always online games clearly don't have a grand history thus far, in terms of reliability. But regardless as to the bumps hit thus far, this concept isn't going away any time soon.

Unfortunately, the negative reaction from players was probably exacerbated by the early salvo of mostly positive reviews from critics that hit prior to launch day. Perhaps most notable among them was Polygon's effusively positive review. Russ Pitts very much adored the game, and reviewed it accordingly, awarding it a 9.5. But he also did so prior to launch, a fact that people have complained about given the apocalyptic server situation. But then Polygon did something interesting; it changed its score. From a 9.5, the game tumbled downward to an 8. Then, as the server issues continued and game services were cut in the hopes of salvaging some server availability, the score dropped to a 4.

Understand, this is all completely in keeping with Polygon's oft-stated editorial policy. At the outset of that site's launch, its editors spent ample time explaining how they planned to review games. In dictating that policy, Polygon made it clear that it believes as games continue to grow and evolve, so too must the review process. Games of the modern era are not static, in that service issues, DLC updates, and patches can all radically change how a product performs. The goal with this system was to allow editors to "bump" (their parlance, not mine) scores depending on how these factors come into play post-review. Each of the updates to SimCity's review have come with detailed explanations from Pitts laying out why these changes have come to pass.

This has been a source of much controversy among those who follow game reviews. Granted, that's probably a fairly small demographic overall, but as someone who has something of an investment in how video game criticism evolves over time, I've been nothing short of fascinated by it all.

The thing is, adding a degree of flexibility in reviewing isn't new. This is something Jeff, myself, and countless others have talked about at various points over the years. In fact, GameSpot has been providing updates to its reviews for a while now, taking into account DLC and patch updates where necessary. Kotaku also does a version of this, with its "Yes/No/Not Yet" scale, which updates depending on how issues with games are corrected over time. My point is, this is a conversation that's been happening in various forms for quite a long time, and it's still something many of us think about with great frequency. Especially Jeff, who maybe spends more time thinking about review policies than is really healthy.

I won't speak for what Giant Bomb's editorial policies could be in the future, since any changes would have to be discussed by us as a team, and decided at higher levels than my own. I will say that for my own purposes, I don't think I'll ever feel the desire or personal need to change my SimCity score. Granted, my approach to reviewing SimCity may have been different than others. I looked at the server issues as one complaint among several, and decided that based on its troubled launch, not to mention the problematic philosophy behind the always-online model, it would be worth dinging the game for these problems. I also made mention that those problems could just as easily go away within a few days. So far, they haven't, but that's not really even the point.

The point is, I'm done with SimCity. I spent many hours playing it in various states of functionality, and by the time I had seen everything I needed to see, I had a set of points I intended to make. Once those were made, and a score was settled on, I stopped trying to play SimCity, resolving myself not to touch it again until the servers completely cleared themselves up. And even if/when they do, I won't be changing the review. Considering that we have no shortage of ways to cover post-release content, through various forms of video, news, and podcast coverage, the idea of having to go back and change a review after some post-release fixes strikes me as overkill.

At a point, it feels like we're coddling these products. I am constantly appreciative of that fluid nature of games, the one that lets improvements and changes find their way into the products long after release, but I'm also of the belief that a game's launch should be indicative of a developer's best effort. When you first put a game on store shelves, you should be putting your absolute best foot forward. That game should be as close to the vision you have for a functioning, entertaining product as you can possibly get, because otherwise, why are you charging people for it? Nobody wants to play an unfinished game, and the increasingly lax attitude from developers and publishers in recent years, one of "put it out and we'll fix it later," is one I just haven't ever been able to get behind. If we're just running back to update these scores every time a developer makes a significant bug fix post-launch, at what point do we draw the line between keeping our audience informed, and bending over backward to accommodate games that launched busted in the first place?

This was Polygon's review of SimCity. Technically, it still is, though the updates have dropped the score all the way down to a 4. Talk about a rough swing.

But that's just my argument against the idea. I'm not eternally declaring against adjustable reviews, but in this conversation we're having, my feeling is that I don't necessarily want or need this kind of flexibility when I'm reviewing a game, because I don't need it from the reviews I read. Of course, my personal disagreement does not make idea behind actively adjusting scores an ignoble one. Polygon isn't doing this because it just wants to be different, I don't think, but rather because it just wants to make sure its readers are able to make the most informed decision possible at any given moment. I expect that's why they opted to review the game prior to launch, as well. In this case, they might have been better served waiting, since it would have negated some of the need for score changing, but when your system is already built to allow for such a thing, there's perhaps little reason to worry about it.

I also wonder if Polygon would even be getting this kind of guff from people if they didn't use scores to begin with. But that's another, longer topic of conversation, one I'm already exhausted with just thinking about it. So it is perhaps enough to say that while I don't necessarily agree with everything Polygon's done in handling its SimCity review, I absolutely support their right to review games how they see fit for their audience. Just because I believe my scores and reviews should remain static doesn't make me believe that everyone should just feel the same way. What works for me doesn't work for everybody. While I believe that the time-tested method of reviewing a game and letting that stick is still completely reasonable for most, I'd like to think that experimentation with said methods should not only be allowed, but encouraged. Everyone is different, after all, and one editorial voice and direction certainly won't suit every consumer.

Talking about these kinds of editorial standards is never a bad thing, and while it's unfortunate that it took SimCity's launch being mucked up so spectacularly for it to happen, I'm just glad we're having these conversations. Not even a few hours after Sliwinski posted his editorial, he and I were chatting it up about editorial policies and flexible review standards and all that fun stuff. These are great conversations to have. People are saying thoughtful, interesting things about the nature of game criticism and how it evolves with the technology, and I love that. After all, if we don't talk about it, then nothing changes. And if there's one thing SimCity has taught me over the years I've spent playing its various incarnations, it's that stagnation is maybe the only thing worse than failure.

--

Just as an FYI, I'm on vacation all this week, and as such, our own Jeff Gerstmann will be handling Sunday column duties next Sunday. See you all in a while!

--A

#1 Edited by goofybandit (2 posts) -

I'm glad I didn't buy the game on launch day.

#2 Posted by OTTO14 (18 posts) -

I have had no issues playing it

#3 Posted by RustySanderke (117 posts) -

Played a ton of SimCity 4 this week. It will probably outlive the new SimCity.

#4 Edited by paulunga (2104 posts) -

What do you mean by "still"? All the SimCity stuff happened this week. It literally hasn't been an entire week yet since all the online trouble started. People saying this is as bad as Diablo 3 are out of their mind. That's not even possible, temporarily speaking. Come back to me in a week or two.

#6 Edited by ripelivejam (4615 posts) -

<p>yes yes polygon has kocked many a knob</p><p>edit: and yes this seems quite worse than the D3 fiasco. i only had like one time where there was an issue personally, tihough YMMV i suppose. they didnt have to strip features out of the game, had numerous issues of lost data (to my knowledge), or have i where the game was simply enirely inaccessible to many over a period longer than a week.</p>

#7 Posted by Homelessbird (717 posts) -

@otto14: Then you are in a very small minority

#8 Edited by DaBuddaDa (290 posts) -

Played a ton of SimCity 4 this week. It will probably outlive the new SimCity.

Me too. I booted up 4 and thought "Hmm...yeah, this'll do."

#9 Posted by ripelivejam (4615 posts) -

btw phone editing seems borked (adds those p tags)

#10 Edited by thugg1280 (90 posts) -

Remember

the Good

Ole

Days

?

when games came out and worked?

#11 Posted by Grilledcheez (3955 posts) -

I had fun playing it just now, but then I lost connection to the servers and figured I may as well just quit since I don't know when they'll be back up.

#12 Posted by iNShadows (8 posts) -

I'm always glad to hear from Alex. His years of experience lends a sense of wisdom and authority on the subject.

#13 Posted by Max_Cherry (1151 posts) -

They didn't expect this volume of players. Don't they know how popular the Sim City name is?

#14 Posted by Levio (1785 posts) -

"EA has become our favorite punching bag of late. Thanks to a series of strange, sometimes blatantly consumer-unfriendly decisions over recent years, we apparently don't like EA very much these days. Actually, I'm really quite fine with them, but I recognize them for what they are: business people."

- Alex, as of 7 days ago

#15 Posted by Sackmanjones (4784 posts) -

Really unfortunate situation. I actually think the game is quite excellent but this is inexcusable. I've been able to get a good amount of playtime in but I know some people haven't. EA had the chance to really step up and prove people wrong. They failed miserably.

#16 Edited by MarkWahlberg (4606 posts) -

Never understood the love sim city got. Pharaoh/Children of the Nile was so much more fun.

#17 Posted by Chibithor (574 posts) -

Remember

the Good

Ole

Days

?

when games came out and worked?

No

#18 Edited by ripelivejam (4615 posts) -

i WANTED to grab sim city 2000 on gog but turns out it's a dosbox version and not the win 95 or whatever one. i have a copy but i cant seem to get it to run under win7.

fond memories tho :)

#19 Posted by MildMolasses (3228 posts) -

Played a ton of SimCity 4 this week. It will probably outlive the new SimCity.

I opened up Cities XL and decided to pop into St. Penisburg and see how the citizens were doing. Turns out, pretty OK

#20 Edited by Nightriff (5257 posts) -

Another great article, enjoy your vacation Alex

#21 Posted by Sgtpierceface (639 posts) -

Polygon are such shitty knob cockers. Fuckers.

You're the worst kind of person.

#22 Edited by beard_of_zeus (1705 posts) -

Excellent article, @alex!

You pretty much summed up my feelings with these couple sentences near the end:

When you first put a game on store shelves, you should be putting your absolute best foot forward. That game should be as close to the vision you have for a functioning, entertaining product as you can possibly get, because otherwise, why are you charging people for it?

Reviews seem to have diminishing returns the farther you get out from release, so something like Polygon's "bumpable" score don't make a whole lot of sense to me. A month or so out (or maybe even less), potential game buyers with probably just be asking friends/peers/forum goers, etc. who have played the game what their experience is. And along those lines, how long will Polygon be watching over SimCity and adjusting the score? I couldn't find anything in the review policy that you linked that specifically goes over that. The closest thing might be:

We do not guarantee that we'll be able to do this with every game, and whether we do so or not is solely at Polygon's discretion.

So...I guess as long as they feel like it? But, to not be completely negative, props to them for trying something different, I guess. Hopefully their actual review policies are as fluid as their reviews, maybe they'll end up switching things up if they find this method doesn't serve their readership as well as they wanted.

#23 Edited by RazielCuts (2982 posts) -

Actual paying customers getting shafted because of pirates, if we have to hate anyone lets hate them!

#24 Edited by johnLongview (133 posts) -

@ripelivejam: ... are you saying that this is worse than the D3 launch because (among other points) SimCity "was simply enirely inaccessible to many over a period longer than a week?" The game actually came out on Tuesday (or late Monday night, depending on your timezone.) So, you know, 5 days ago. I understand the... enthusiasm that people seem to have in overstating things when it comes to EA, but seriously.

#25 Posted by Sterling (2604 posts) -

#26 Edited by ripelivejam (4615 posts) -

i also keep hearing stories cities xl runs like shit due to memory leaks? how substantiated is this? i have an aging core2duo, 4gb ram, hd6850 and despite my best instincts have really wanted to pick it up on steam sale.

#27 Posted by Hailinel (25203 posts) -

Polygon's adjustable scoring policy isn't a solution. For the common interpretation of a ten-point decimal scale, going from 9.5 to 8 is next to meaningless. Dropping from an 8 to a 4 is going from "This game is great" to "This game is garbage" and stinks of damage control on Polygon's part.

Maxis and EA screwed up on this one and they deserve the criticism that they get, but Polygon has done nothing but confuse the situation while making themselves look incompetent. They've effectively rendered their SimCity review score meaningless; when the server issues are finally resolved (whenever that is), does that mean that they'll bump the 4 back up to 9.5? 8? 7? Is there any reason to even give a shit about the review at that point? They've already rendered it a punchline. And what am I supposed to do the next time Polygon reviews an online game? Take it at face value, or wait for the score to inevitably yo-yo while the server kinks are worked out?

#28 Edited by bennyboy (329 posts) -

Great piece, Alex.

#29 Edited by rcath (384 posts) -

STOP SHIPPING BROKEN GAMES>Q!!@

#30 Posted by fisk0 (4406 posts) -

@rustysanderke said:

Played a ton of SimCity 4 this week. It will probably outlive the new SimCity.

Me too. I booted up 4 and thought "Hmm...yeah, this'll do."

Same here, I used to be much more fond of SimCity 2000, but booting up SC4 last week, I realized it actually implements most of the ideas for the new SimCity that actually seem sound - like regions (without vast areas of unavailable wilderness inbetween the cities) and curved roads(!), while remaining easy to pick up and play. I've spent a good 10 hours with it in the past week, and that's a pretty good chunk of the gaming time I have in a week.

#31 Edited by MeatSim (10927 posts) -

I mostly didn't get SimCity, becuase I wasn't sure how it would run on my machine. But I really dodged the bullet this time thanks to those concerns.

#32 Edited by franizarduy (10 posts) -

Still, you should ve given it a lower score review in the first place, as you claim to express, games are analysed as a whole ,simcity is a failure as an always online product, i dont care if it has fantastic production values or technical wonders, because all of these features are meant to be online, so if this premise Doesnt work the game is broken. Personally i prefer kevin van ord's review cause i think is more consumer friendly.</p>

#33 Posted by johnLongview (133 posts) -

@hailinel said:

And what am I supposed to do the next time Polygon reviews an online game? Take it at face value, or wait for the score to inevitably yo-yo while the server kinks are worked out?

Or, you know, read the text?

I'm wondering how much of the hullabaloo regarding Polygon's adjustable scores is related to the importance that people seem to place on the number given. I've never heard such ire when "updates" are added to reviews and don't affect the score, even though they may color the review substantially.

#34 Edited by ripelivejam (4615 posts) -

i would like to analyze my game hole too :\

#35 Posted by EnduranceFun (1114 posts) -

I've never visited Polygon and I think I'm going to actively avoid it after reading this article.

#36 Posted by DefaultProphet (550 posts) -

I don't like that Polygon kept Diablo 3's 10/10 the entire time it was having similar issues. But they'll knock down this 5.5 points. Just seems like more Blizzard/Valve Master Race favoritism

#38 Posted by dropabombonit (1492 posts) -

Great article and good to see a considered piece about the review scores issue. Shame it took EA royally botching up this SimCity launch to get people thinking about it

#39 Posted by dr_mantas (2016 posts) -

If you expected the servers for an always online hugely popular game to be working from the start, you are, if nothing else, naive.

It has taken longer than most would expect, and I feel for the people who spent money on it.

Next time - don't. Simply wait.

#40 Posted by DanTheGamer32 (226 posts) -

@defaultprophet: maybe that has more to do with that they weren't hosting their own site when that review was out.

#42 Posted by momofire (24 posts) -

@johnlongview: Read his post again. Saved progress lost for some paying customers, Features turned off in a weak attempt to save server space, playtime caps, and of course still not working this far after launch. Diablo was screwed for 24 hours. The difference is huge.

#43 Posted by Spankmealotus (300 posts) -

The issue that should be getting discussed isn't whether or not reviewers should go back and change their review later. I think the main thing when discussing the reviewer side of this issue is if it's okay for people to put up reviews of games based on a situation that isn't indicative of the experience the end user will have. It makes absolutely no sense to me personally. As purchasing advice that a review is meant to be, that advice should be based on the experience the purchaser would have upon purchase, and not some experience they may have at some point down the road. This is even more important with the point Jeff made about a review only really being relevant for about 21 days.

Of course this is only an issue here because of the online requirement making the fact that the review is based on a "retail copy" or not irrelevant.

TL;DR Putting up reviews for games that have some kind of "service" attachment to them before release is a terrible idea and, for me, discredits the opinion of said reviewer.

#44 Posted by skyline7284 (508 posts) -

I've played 17 hours of Simcity 4 this week.... Man it's so good you guys.

#45 Posted by Homelessbird (717 posts) -

Played it yesterday for 3 hours straight, and today I had a session with a friend for 6 hours and not had a single problem with it. So it seems like they are fixing things which is a good thing for SimCity (it's not a terrible game)

But everyone hates EA this month, so any chance they get to bitch and moan is a chance they're gonna take. Same thing happened with Diablo 3, people hated on Blizzard, stuff got fixed and now they're right back at sucking Blizzards dick all the way to the root. Same will happen with SimCity.

Predictable hivemind

Wonder what score the game is gonna have when it works as it should.

Or in a few years.

Your good experience does not invalidate the vast majority's negative experiences, or mean that the server problems don't exist.

And as far as I know, Diablo 3 didn't lose hours of progress because of server instability. Nor did Blizzard refuse refunds to people who were upset. Or ban them from their services for asking for refunds. Or come out and publicly say "you can't play our game because the people who ARE playing it are just having TOO MUCH FUN TO STOP!"

Whatever you think about EA and this game, they have screwed up this launch BIG TIME. And they've done it in a way that betrays a very corporate attitude. And with a service model - always-online DRM - that gamers hate, viscerally. They would have had to make a good case for it with their launch to NOT get some measure of this rage, and they instead went in the other direction.

But go ahead, blame the reacton on a "hivemind" or "EA haters." That seems like a really thorough analysis.

#46 Posted by Milpool (507 posts) -

So once (if) the server issues get worked out, is Polygon going to raise its score?

That just seems weird to me. I understand what they are trying to do, I guess, they want to give the most accurate score that they see fit of a game, at a given time. But where does it start and stop? Shouldn't they update every game's scores whenever any kind of patch or change is made to a game?

It just comes off as looking kind of shoddy, like "oops we reviewed this game too early" or "oops we can't make up our mind", especially when they present the history of changing scores for a single game.

#47 Posted by ripelivejam (4615 posts) -

also that header image just looks so damn slick it still really makes me want to play it based on that alone. why couldn't they just apply the 3d engine to sc4? :(

#48 Posted by wumbo3000 (999 posts) -

I love this feature. Great article Alex.

#49 Posted by Excast (1034 posts) -

I understand that Polygon wants to keep readers updated with the current situation, but they did people a disservice by posting a review before the game could be stress tested. No game that requires you to always be online should be reviewed before that online system has had a chance to be tested.

And for the people making the MMO comparison. Sure, MMOs launch issues don't usually anger as many people. Do you know why? The very nature of the game. The Sim City and Diablo franchises do not have histories of requiring you to play with other people or to be online. So when a franchise is fundamentally changed in order to justify it always needing to be online, and then things crash and burn to the ground, fans of the series are going to be justifiably annoyed.

At the end of the day, customers want to be treated with a basic level of respect and decency. Instead we are being used as beta testers for products that were clearly not ready for primetime.

#50 Posted by Draxyle (1888 posts) -

My biggest problem with polygon's score is simply due to the fact that their first score was dishonest. In trying to beat the rest of the internet to the table, they put out a review that was not indicative of what the game would be at launch and without any warning of any kind. A whole lot of people bought the game because of that near-perfect score, and it's far too late to change an arbitrary number for a lot of those people. A changing review score is okay for an MMO-like game, but they botched that launch score as much as EA botched their launch servers.

And about EA. I'm getting quite annoyed of them spinning every apology into a personal brag of some sort.

The good news is that SimCity is a solid hit in all major markets. The consensus among critics and players is that this is fundamentally a great game.

When you are apologizing for your huge mistake, this is not what I want to hear. This feels like a statement that went through the hands of five hundred marketers to try and make this situation sound as good as possible, and they do this absolutely every time they have a PR disaster (which is quite often, I might add).

For once I'd like them to speak to us like human beings.

This edit will also create new pages on Giant Bomb for:

Beware, you are proposing to add brand new pages to the wiki along with your edits. Make sure this is what you intended. This will likely increase the time it takes for your changes to go live.

Comment and Save

Until you earn 1000 points all your submissions need to be vetted by other Giant Bomb users. This process takes no more than a few hours and we'll send you an email once approved.