Joystiq doing away with review scores

Joystiq provides a lengthy explanation as to why they chose to focus on the content of the printed word rather than the reduction of said content to a "star" system:

Joystiq isn't scoring reviews anymore and here's why

http://www.joystiq.com/2015/01/13/joystiq-isnt-scoring-reviews-anymore-and-heres-why/

Thoughts?

Jeff Gerstmann put some interesting thoughts about it on twitter when Joystiq's announcement hit. Personally I don't need review scores and applaud Joystiq (and Kotaku) for ditching them. I do believe reviews still have a function though. What I get out of a review is different than what I get out of a quick look.

I don't agree with everything he says, but right now, I agree with Jeff on the issue of review stuff. It's a half step. There is a point where reviews don't make sense, anymore. The game is given a quick look, which shows it off in a raw, unscripted moment. If it crashes, bugs out, looks great, has a bad framerate, etc., it isn't polished footage and the commentators reaction is real. It isn't an edited wall of text that has been proofread 14 times. People don't read reviews anymore, anyway. They watch video on Youtube or GB or somewhere, go to Metacritic (sometimes) to look at a number, and then decide. If they wanted to be "revolutionary", they should have just killed reviews period in favor of a different flavor of content that still conveyed their opinions.

Reviews are still relevant for when you want a comprehensive look at a game. Videos are great for getting the feel of what a particular title is all about, almost like a vertical slice would. A Quick Look, no matter if it's 20 minutes or over an hour long won't let you know how things pan out from start to finish. I think many of us at one point or another must have purchased a game because of a Quick Look, only to later find out there wasn't much more to it than what we saw in the video. Hitman Absolution is a good example of a game that can demo really well or really poorly depending on what level you show and that single level doesn't tell the whole story. For that purpose I think written reviews are still really great and I don't think they necessarily need a score attached at the end. I've always been of the opinion that if you're a good writer the review will tell the good and the bad.

Not that I have any particularly strong feelings against numerical scoring one way or another.

Have to say, there is something natural you get from a quick look that scripted previews and clips just don't offer. There is a feel for the pacing of a game, and it comes across as more of a genuine look at what a title is about. It's also very unique, in that I don't know of any other site that offers this sort of content.

When it comes to reviews, however, I actually do think there is value in having a scoring system. I appreciate the x/5 or x/10 marks that are provided, as it lets you see at a glance if the game is (in the eye of the reviewer) pretty good, average, or a steaming pile of manure. Of course, that doesn't mean to say the reviewer, or the score, is correct. Nor should it suggest that the person reading the review has to agree with that final score.

For me personally, I like to read reviews for
a) Games I am already really interested in
b) Games with unusually or unexpected high praise (perfect scores)
c) Games which get terrible write-ups, and the reviewer is picking their way through the carnage!

Having a scoring system in place lets me see at a glance if a review is generally going to fall into one of those categories in advance of reading the review itself.

I've heard Jeff talking about this particular topic in the past on one of the Bombcasts. The question that really comes in with a review is "Is this worth my time / money". The problem GB staff (and pretty much most game journalists actually) have is that their valuation of time and money is not exactly in line with the general public's.

My only hope is that if GB were to change up how they review a game, that they replace it with some other measurable. From reading Jeff's tweets, I don't know that would be the case.

What I see is that video game sites are deciding whether their reviews from a buyer's guide to a game critique. What does a site's game audience want from a review?

I see the comments of game reviews filled with people who have already bought, played, maybe beaten a game and they would either agree or disagree with the reviewer. I think these folks are not interested in a buyer's guide for a game, but a game critique.

For me, a quick look is my buyer's guide, I use reviews to think about things I may not have noticed about the game previously or just read interesting thoughts about the reviewer's experience with the game.

I think Joystiq is probably responding to its readership probably expecting more of a game critique rather than a nebulous star system.

I don't like reviews without a score. A score summarizes how good the game was in the eyes of the reviewer. I want to know that. You can't communicate that without a score. The final paragraph is not the same; it's often written as a conclusion, not a verdict of whether the pro's are worth the con's. It happens often that a review sounds rather negative tonally, but then goes on to receive a very good score, or vice versa. Leaving the score out would just mean I'd have to go by the tone of your review, and judging by the scores that are given: that's a set up for miscommunication.

Also, Kotaku's approach totally sucks. They have/had (haven't been there in a while) a 'play' or 'don't play' recommendation. Which is pointless. It needs to say 'Play' on every single game. Because if it says 'don't play', then you're telling me no one would like that game. Which is almost never the case and your review is not credible. Which means the entire point of having that stuff is useless.

As more and more people get their buying advice from video, the place for the written-word form of the "buyer's guide" seems to be shrinking. I guess people still care about Metacritic, though? Anyway, what I'm trying to say is that when you take out the measurable part of a review, you might as well call it a "first impressions" piece or whatever, since it will probably fill in a different gap in information about the game (one that I'm definitely more interested in, for that matter).

I think it is dumb of them and I wouldn't be surprised if they reintroduce the review score within the next year. Me personally I will quickly look at the score and read the review if I was curious why they gave said game the score, now I just won't even go into their reviews.

it's really cool that they are getting rid of scores but seems bad for their business. maybe their audience like to read more than most generally do?

honestly, i like quick looks a lot but they aren't good purchasing guides for me. i like the guys and i like watching them f around with a game for an hour or so but games and developers deserve better than to be judged that way. finish the game or spend like 10-20 hours on it before telling ppl something sucks or not. i just can't get behind the whole "if i don't like the first hour that much, i don't like the game" approach. there is a huge difference of tone whenever they actually review a game vs. just a quick look.

In a perfect world I'd like a video quick-look (not a video re-hashing of a review like some sites do) in conjunction with a written piece that explains all the things the "raw" video QL misses (which can be a lot).

I see the need for both in-depth review, whether written or video recorded, and the quick summary. One caters to those who want more information and one for those who want less. But boiling a review down, especially a well-thought out and written review, to a series of stars or numbers seems oversimplified. Ask yourself this; would you make a purchase based solely on numbers or stars? Remember that you are planning on spending your valuable money and time on this game. Why not start off right by spending time reading the review?

Stupid and meaningless.

How daring!

Scores are pointless. They're supposed to be the end-all be-all "summary" grade but they fail at that. "WHY DID YOU GIVE GTAV A 10 BUT THE LAST OF US A 9.5????"

I'm ok with more vague scores like 5 stars or letter grades but the actual number system is absolutely terrible.

In any case, what's said is better than one or two characters typed in large text at the end of the page. As for the "readers like a quick summary" argument, what's wrong with short bullet points? If you're too dumb/lazy to take the time to read a couple of lines of bullet points then....uh.........

Hopefully more follow suit.

I'm not against review scores in principle, I just don't think game reviewers can be trusted with them. If you're going to score practically everything that gets put in front of you between ~75 and 95 it makes them redundant.

We currently have a games press which is, for the most part, absolutely terrified of calling a bad game a bad game. Perhaps they will feel more able to do so with words alone than when they have to go out on a limb and assign a cold, hard numerical value to a game.

More outlets simply need to adopt the 5-star scale, as it's the ideal method of scoring I feel without it being too granular, and I think reviews score still hold some worth. It sets the tone of the review and helps compliment the text overall I think. Plus it can sometimes make for an easy go-to of saying, like, ''I think this is a 4-star game'' when you want to generally sum up a game's worth or quality.

I'm all for this becoming a thing if it means that people have one less thing to argue about.

I dunno. While I agree with the general premise that review scores are generally pretty shallow and useless as a tool for deciding whether or not to buy a game, I feel like scores still need to exist if for no other reason than to be used as an organizational tool.

When I'm deciding whether or not to buy a game, I want to hear from a variety of sources, and I try to seek out both good and bad opinions on the game. For this purpose the review score is useful, since it's a shorthand way of figuring out which reviews are which. If the score didn't exist, I'd need to read each review individually to sort the positive reviews from the bad. This could be a problem if there's a generally well received game and I'm looking for the one or two mediocre/bad reviews on it. With a score, I can find those much more quickly and see if I think the complaints are valid. Beyond that, I'm probably just going to ignore the score and read the actual review.

I like ratings. They tell me everything I want to know up front or I can read further if I want to know more. I don't see the problem.

Although Quick Looks and similar video content provide a dynamic and entertaining look of a game, I still prefer to consume a more evaluative piece of content in addition in order to get a balanced view of the game before making a decision to purchase to game or not. Whether that piece of content would be a review or something else, I don't care.

That said, if GB's Quick Looks would always - or at least more often - feature one person who has played through the entire game before Quick Looking it, these could of course be combined in a more structured way. But the Quick Looks as they are today do not provide enough information for me on their own.

My only problem with alll this is that while reviews are informed and generally researched, quick looks or let's plays are a lot of times just dudes jumping into a game blind. In more complicated, complex games, the lack of knowledge and time spent learning systems isn't accurately reflected by a quick look.

That said, it seems like Joystiq isn't doing away with reviews entirely, so that's awesome. I look forward to seeing how their experiment will work out, though I have some crass ideas about how little the Internet at large generally likes to read.

I swear they tried this before. Like, a few years back and then all of a sudden they had scores again. Maybe it will take this time? Probably not.

@hassun: I agree whole-heartedly with what Jeff says, especially if you consider some of the practices we saw last year. The fact of the matter is, press reviews aren't always representative of the final product, and that's an issue. If the consumer can find more honest coverage of these games, then they will go elsewhere. We've already seen this happen as more and more gamers switch to Let's Plays and first impression videos to decide on whether or not they want to buy the game.

I personally think there is a place for full reviews and initial impressions, but I'm probably not part of the targetable mass.

@zevvion: If you can't tell whether a reviewer liked a game or not based on the body of the review then that's a bad review, and adding a score won't make it better.

I like seeing review scores go away, mainly because of metacritic. It's insane to me that any kind of business decision would be based on such arbitrary, inaccurate and ultimately useless tool. But apparently it happens a lot. It's good to see more outlets give up on metacritic. That said, out of all scoring systems 5 stars seems to me like the least offensive and the most sensible.

@sparky_buzzsaw: @grtkbrandon:

Exactly. Even though it's true games these days can change a lot post-release or post-review I still wouldn't want reviews to disappear. There is a big difference between quick impressions of a game and someone having thoroughly played the game and giving you his or her thoughts on it.

Nice move. Get rid of the q while you're at it!

Seriously though, I think some form of review is still viable these days. Sadly, it would mostly have to be about the technical side of games, since that one suffers the most (games straight up don't work sometimes) and people NEED to know of such issues before they buy this stuff. Treating this review thing like you're a movie critic, talking about 'narrative' or 'performance by x actor' or whatever buzzwords get thrown around these days.. fuck that. I want to know if a game works or not, THAT is the most important part.

Next on the list would be how it plays and then maybe, maybe you get to write about freaking ludonarrative dissonance.

Edit: So of course I misspelled the buzzword :D

I give this post a 8.79

A 74 RTS game is not necessarily worse than this 79 shooter, though it does strongly imply it. They are both 4 star games. I do like some sort of vague score so you can see off the bat if a game is good or not and if it is in line with my general expectation.

@superfriend: I don't think games have fallen quite so far that the main purpose of reviews should be determining whether they work or not. That should be a given, and any game which falls short of that low bar should be getting absolutely destroyed at review (as would happen in any other medium). We should really be aiming a bit higher than that.

Reviews are only relevant when purchasing a game. A review for Halo MCC or Assassin's Creed: Unity would read differently if played then written today rather than during the launch week. It can be incredibly tough to find accurate purchasing advice based on reviews and scores only from launch.

Video game launches don't represent the game 1, 3, or even 12 months after release. So why invest so much time and effort into a review when it poorly represents what you're trying to convey?

You need to have coverage of a game. Videos, reviews, discussion. If you review Halo MCC and never touch it again, the game is broken and not worth purchasing. But if you play it a month or 2 later everything is fixed (well most everything) and you can experience the game much more positively.

Removing the score is a smart decision because it stops the typical comments focused on scores and forces discussion. But the review itself is becoming much more flawed.

This. In the MCC quick look, we saw the network issues that caused multiplayer to be nearly unplayable or at least difficult for weeks. When you go look at reviews, Eurogamer, IGN, Destructoid, Joystiq, etc. all gave it +80%, without taking the multiplayer into consideration. So many outlets didn't acknowledge it in the reviews. They simply played some matches, some campaign, and judged it on that.

I think that reviews just aren't always a good tool for some outlets. Everyone who runs a video game outlet wants to do reviews. But honestly, most of us hate 90% of the reviews that outlets publish and have 1-3 that we trust, and there's likely overlap there. So, I honestly don't think that there's a need anymore for everyone to review games. It's just loud noise all at once at this point.

This seems like a pointless decision....

There's really no point in joystiq reviews now...

I can't tell you the last review I actually read, so personally I'd be in favor of doing away with them entirely. Between youtube, twitch, social media, and quick looks, I always know what games are good before I buy them.

@spraynardtatum: I don't agree that there is "no point" in a Joystiq review without a score attached to it. A well-written review will put forth an impression of where on the "quality scale" a reviewer would place a game, despite it not having a score. I do think the 5-star scale works well, but it's not absolutely mandatory.

Scores are sometimes very silly for the myriad reasons people have already posted. I can't count how many times I've read a review where people complain in the comments that the written review seemed to them to imply a 6 and the final score was 8, or some such.

If reviews go away I want to see more written critiques. That's what I say. I love reading reviews (basically just Jeffs) but I'd love them to be even more critical. Reviews always just turn into, "they made a competent/incompetent version of this style of game". Reviewers are mostly lazy in the way they approach them and it's painfully obvious when there isn't a lot of thought put into their opinions. After the third or fourth marketing bullet point they become mind numbing.

I want some good meaty critique. More of the gaming press needs to get their hands dirty. It's all gentle fluff that borders on promotion.

I think I'm with you. Reviews aren't needed anymore. I would say that comes from a combination of all the other avenues that you can find out about games now and the problem of reviewers just going through the motions. The passion isn't there. They don't care about what they write and it shows. Where's the gaming equivalent of Roger Ebert?

Most reviews are shitty.

If they called them critiques and approached them as such than they don't need to come out on release day, we'd see much better writing, and we'd push away from the marketing nature that most gaming sites cling to.

Interesting article. I can appreciate the points made about parsing out intangibles like fun into a score and games like MCC being worse post-launch than they appeared when reviewed, but this sounds like them trying to justify not doing something they don't want to do. Which is okay, I guess, but if they're not going to do that, then what are they providing? I'm admittedly not that familiar with Joystiq, but a quick perusal of their site does not impress, given that Rainbow Six: Patriots tops their "Recently Released" list. Um, no. It seems to me like they just don't want to have to defend or really support their review scores, and that's pretty weak. I know the internet is often shitty, nitpicky, and wearisome, but as a reviewer, if you expend the time and effort to reach a well-thought-out conclusion about something, you're probably secure enough in it to not let shit like some rando not liking the score you gave something bother you. It just feels like a cop-out.

On this note, I'm not here on GB for their reviews. I'm here because this is an entertainment site (which is damn good at it, BTW). I'm not saying that the site lacks worthwhile insights, but they don't tend to take things all that seriously, and that's perfectly fine and refreshing in a world that too often does. I get the sense GB doesn't really like to do reviews, either, and just considers them a slightly annoying part of what they think is expected out of them as a video game coverage outlet. And I fully understand where they're coming from, because I figure they'd probably rather be producing content they want to on their schedule rather than getting that review of x game up by its launch/when the embargo lifts. Also, as someone who's overly verbose and often has their thoughts skipped over by those who don't feel like taking the time to read them, I can identify with spending time writing something out knowing that a bunch of people will skim over it, and I don't even have that distracting nugget of a "verdict" summation after my posts. In other words, I get why the GB staff may feel reviews are a waste of time. Given that the other stuff they produce is generally fantastic, I'd be 100% fine with them choosing not to do reviews if they felt so inclined. They've earned that, but I appreciate that they still take the time to do reviews all the same.

Anyway, I think reviews should have scores tied to them, but they need to be more than that. Mostly, I want reviews to explain a game's strengths and weaknesses in detail, because odds are that the reviewer and I don't have the exact same tastes, so something that they may like/dislike may or may not be something I care about in a game. And given that many reviewers these days tend to go out of their way to find something offensive/controversial in the interest of self-glorification or creating clickbait, it's good to know when a score can pretty much be invalidated in my book if it's based on shit that does not matter to me. That's another topic, though, and for the record, as long as reviewers are explicit about their reasoning (no matter how flawed/whiny), I'm good with it even if I highly disagree. I can recall many times taking more away from the content of the review than from the score assigned at the end (which sometimes is arguably incongruous anyway).

The actual review scores to me are useful info, but not the end-all. I like having an aggregate baseline via metacritic, because while I don't only buy games that review well or flat-out avoid ones that don't, having a general idea of what to expect is nice. I know what I like in many cases, but if I'm on the fence, I can be swayed one way or the other. And in my opinion, the less increments in a score, the worse it is. "Play/don't play" is incredibly lame and simplistic. I'd even contend as Joystiq does in that article that the five-star system (which is used here) is not specific enough, because there's a huge difference between an 80 and a 60 in my book. And for a game that a reviewer may really feel is 70ish, having to choose to underrate it at "60" or overrate it at "80" is kinda silly. If it were up to me, review scores would all be x/100 of x.x/10. I think that level of specificity naturally lends one to more deeply consider many aspects of a game, whereas more vague scoring systems naturally lend themselves to "eh, I give it ____", which isn't exactly the level of deliberation I'm looking for from a review.

Ultimately, I'm glad reviews exist, and I hope people don't follow suit with Joystiq and stop scoring games. GB is the lone exception if they want to be, because GB is alchemy and will be whether it puts out proper reviews or not. I just don't like the idea of your average yahoo video game outlet thinking they're "too good" to bother with what probably should be a key feature of most video game sites. I give Joystiq's choice to no longer do reviews and reasoning for doing so a 3.8/10.

@spraynardtatum: I would be more interested in reading deeper articles about games, like post mortems or like the thing Patrick did on Eve Online. That was fascinating. But reviews, as a purchasing tool, are kind of pointless to me now.

Their reviews are just pointless now....I just....don't see the point in their reviews anymore.

@spraynardtatum said:

I can't tell if you are being cheeky or not.

You say "If reviews go away I want to see more written critiques." but it sounds like just juggling semantics. It sounds like you want reviews to evolve & expand; you might not call it a "review", but wouldn't a "written critique" just be a bigger, better "review"?

If that is the case - I agree. I want reviews to transcend the mostly-rehashed buzzwords they are now into a more-encompassing critique, but I'd still consider them a review. If they include a video component to illustrate some points, all the better. If they have the potential to grow to include things like post-mortem discussions, et al.... sounds great.

Nobody wants to read a review longer than the lengthiest one Jeff has written in the past few years, even I don't, unless it's like Tim Roger's Week with Animal Crossing review. Putting a Kotaku-like-yet-not blurb at the bottom like that doesn't help when there are infinitely other sites that can give me an idea at a glance, which is what I'm looking for.

The problem lies with the big-three publishers who use Metacritic to decide when a developer should receive any bonus at all.
The problem also lies with reviews giving 70/100 to the most broken bad games, and 85 to games that are perfectly solid and enjoyable, long-lived but not of a AAA budget and big brand.
The problem lies with Joystiq giving up their five-star system because of #1 and #2 instead of speaking out about it and starting a movement.

common issue for me that i know others have seen is the almost always disconnect between the score and the actual written review. you'll read the review which mentions a few caveats that may be somewhat worrying but not deal breakers along with a number of neat features and qualities, and then you see the 6/10 or 3/5 slapped on the end which makes you possibly second guess something you were initially interested in. It could be the negatives impacted the reviewer's impression of the game stronger than the review itself let on and thus they're weighing the overall score as a result of it. just seems like something that would appeal to a reader if they actually read the review becomes a turn off when they see that score. then again that could just be fault of impatient "readers", but i do think that number score has a potency on the general readership.

but reviews going away completely i would not be happy with. i would miss five (four?) star shoemaker's eloquent insights.

Without reviews I doubt Brad would finish any games, so his game of the year list would be just Destiny and Dota 2 written 5 times each.

@whitegreyblack: We're totally on the same page about a critique still being a review. It's a small difference in semantics but it's an important one I feel. A critique has more clout than a review. It's a more thought out and thought provoking review that isn't there to sell copies or get a developer a bonus.

But you have to agree that their reviews are pointless now. It's literally what the entire article is about. There's literally no point in their reviews anymore. A joystiq review is pointless.

I've still never seen an explanation for how removing scores actually benefits a reader. It's just less information, and I'd rather have more.

I get what he's saying. How many people regard Joystiq's reviews as good critque of the game regardless of the score? A large number of people don't care what they wrote. They go see the score and leave. Now that there is no score, their reviews are pointless because there is a large opinion that their reviews aren't written well and/or aren't good critiques of the games. So there's no point in them writing the reviews when the reason people go to their site is for scores.

@spraynardtatum: You know what - I am coming around to your way of thinking on this one. A scoreless review... well, in the sense of what we all have come to know within the video game industry, is no longer a review. You can argue it no longer has value as a "review" in the way that we view and consume video game reviews.

Write this up as being the first and last time ever, in the history of the internet, that someone has actually changed another person's mind on something in a forum thread. Shut 'er down. Mission accomplished.

I like review scores personally it gives a context that I enjoy. Still get excited when I see a 5 on HBO on like a 9 + on pitchfork

It's an interesting idea. Not fully sure how I feel about it though. On one hand a score allows me convenience. It also works to help the commercial marketing of a game. On the other, a strictly written review without a score incentivizes someone to read the text and extrapolate the worth of the game based on the reviewers experience. I will say that I agree with Jeff on the matter. If you're going to do away with scores, then maybe do away with reviews; replace them with critiques or something of the sort. Though, you may ask - isn't that the same thing? In a sense, but in this day and age the word review instills a very specific definition in the mind of a consumer. (Text followed by a score) Some people want simplicity and the ability to skip the text and see a number based summation on a product. But for other, more avid enthusiasts, a written experience is worth more. And then there are others who value both. I guess what I'm saying is the reaction toward a change like this will depend on the individual. I do wonder how it will work out for them.

I think Polygon's way of doing scores is a win/win situation.

When the publisher is running ads, you give the game a favorable score. Once the community finds out the game is crap out of the box, drop that score like a rock to make the community happy. The first score is the one that sticks on metacritic, so the publisher/devs are happy. Win/win! Polygon always looks like the good guy to everyone!

If a place does scoreless reviews, I'd like to see Kotaku's implementation used. The Yes/No/Maybe followed by a quirk blurb as to what's good, what's bad, and what's goofy works much better than an arbitrary 3.9756847693474568467396 out of 7.5.

I don't want some internet person's numeric opinion to validate my purchase decisions. I want some random internet person's opinion to help inform me on a purchase I'm already iffy on. If random turn based RPG got a score of 1/5, I probably wouldn't even look at it. Even if, somewhere in the author's wall of text, they stated they disliked perma-death turn based RPGs. Kotaku might give the game a "Maybe" with a "Disliked: Perma-death of units." and if I was cool with that, I'd do some more research.