Great breakdown of the reviews
By the Numbers: 500 Reviews on Giant Bomb
@Marino, as always, you do amazing work. Whodathunk that a guy with a cartoon Bret Hart avatar would be such a stats junkie?
@Encephalon said:
Vinny, the People's Champion.
If I had any photoshop skills whatsoever I'd photoshop Vinny's face onto The Rock giving The People's Eyebrow...and then I guess I'd have to photoshop the eyebrow itself onto Vinny's face.
@D_W said:
I think it's real interesting that for almost everyone 4 stars is the most common score.
It's just because all games are slightly better than average.
Wait, that doesn't make any sense.
Interesting.
Didn't realise Jeff accounted for quite that many reviews - more than Brad and Ryan combined. See the breakdown of stars awarded was interesting. Also the stand outs from which games had the highest number of user reviews and might have been suprising would be Mirror's Edge and Prince of Persia (I presume the cell-shaded one). I reckon the Mirror's Edge number might be so high partially because there were many people who reacted against the 3-star GB review and wnated to show their support for it. Nice work Marino, anyway.
Really cool to see this stuff broken down, as always. I kind of want to see just how much raw data you had to work from to create this, but I fear it.
I have one gripe, and it's only because I do this kind of stuff frequently in my course, the formatting of your graphs bugs me. Every time I look at this stuff I can't help but want to order it correctly instead of having it in random order. Pie charts are meant to start with the largest chunk beginning at the 12 o'clock position and the other chunks in descending order clockwise.
That nitpicking aside, this is awesome. Do more.
Why did people seemingly give up on user reviews circa 2009? Or is it just that today's sequels to those games haven't been around long enough to garner up the same totals?
These charts are really confusing, but they definitely don't make sense. You can't say user reviews either "agree" or "disagree" with staff reviews, if a staff review gives a game 4 stars and a user review gives the same game 5 stars, they are if anything agreeing rather then disagreeing. Same if the staff gave 5 and the user gave 4. Same could apply for 3 stars and 4 stars ("good"), and 1 star and 2 star ("bad"). Not to mention these are just ratings. Two people can give the same rating but that does not mean they agree. To a lesser extent the opposite is true as well.
@boocreepyfootdoctor said:
Why did people seemingly give up on user reviews circa 2009? Or is it just that today's sequels to those games haven't been around long enough to garner up the same totals?
I don't know but it kind of sucks that user reviews don't show on a wiki page unless the staff reviewed the game. So usually there is no point in writing a review since nobody will see it.
@egg said:
These charts are really confusing, but they definitely don't make sense. You can't say user reviews either "agree" or "disagree" with staff reviews, if a staff review gives a game 4 stars and a user review gives the same game 5 stars, they are if anything agreeing rather then disagreeing. Same if the staff gave 5 and the user gave 4. Same could apply for 3 stars and 4 stars ("good"), and 1 star and 2 star ("bad"). Not to mention these are just ratings. Two people can give the same rating but that does not mean they agree. To a lesser extent the opposite is true as well.
I understand what you're saying I guess, but all I did was break it down further. You're basically saying that a 4 and 5 are both "good," so they're equal in terms of "agreeing" that a game is good. What I'm looking at is specifically how good or how bad the game is in the eyes of the staff and the community. That's the point of having stars or points anyway. So, using your example, if the staff gives a 5 and the average user review equals 4, yes they both "agree" that it's a good game, but the staff thinks it's better than the users do, therefore they "disagree."
@Thompson820 said:
Really cool to see this stuff broken down, as always. I kind of want to see just how much raw data you had to work from to create this, but I fear it.
I have one gripe, and it's only because I do this kind of stuff frequently in my course, the formatting of your graphs bugs me. Every time I look at this stuff I can't help but want to order it correctly instead of having it in random order. Pie charts are meant to start with the largest chunk beginning at the 12 o'clock position and the other chunks in descending order clockwise.
That nitpicking aside, this is awesome. Do more.
As for the raw data, I could post the Excel file I guess, but it's basically just a giant table with headers for Game Title, Platform, Review Date, Review By, Staff Score, # of User Reviews, User Average. To get the data, I just had to go to each game's review page. The site already provides an average score on user reviews. The only tricky part was reviews on DLC. I had to manually pick those out and calculate an average since all DLC reviews are bundled together on the game's page.
As for the pie charts, yeah, I thought about that too. But, for whatever reason, I just stuck with a uniform 5-4-3-2-1 clockwise format. And on the giant platform pie chart, I wanted to keep it so that you could see the Sony platforms all together vs the Nintendo platforms all together, etc. I tried to keep the colors in such a way that variations of blue/purple was Sony, red was Nintendo, green/orange for Xbox/XBLA, yellow for Apple, etc.
@gladspooky said:
@D_W said:
I think it's real interesting that for almost everyone 4 stars is the most common score.
It's just because all games are slightly better than average.
Wait, that doesn't make any sense.
Heh. That's what I had thought at first as well. That's the problem if you think of reviews in as objective things, which they very much aren't. I think this data shows more that review scores are more just metrics on how much that individual reviewer liked the game. Which is essentially what Jeff has been saying about reviews for years.
Fantastic work. I can't believe it's been four years. That's craaaaazy.
I totally forgot Andy did a review.
@Marino: Hey Marino, i love you and i sincerely appreciate all the work you clearly have put into this awesome post, so thanks for that,
HOWEVER,
i have to point this out: The 3D pie charts are a terrible choice for displaying these data. 2D pie charts would have been slightly less terrible, but still a bad visualization. A horizontal stacked bar chart would be more appropriate in this case, like so:
The bumps chart is also a worthwhile idea here:
I wonder how this would compare to the other reviewers' versions of this chart.
@BirdkeeperDan said:
I am amazed people can be confused by pie charts.
Actually it's very easy to get confused by pie charts. They're just not very good, and often used where another form of visualization would be more appropriate to the data. In many cases, even a simple table might be a more effective communication of the data.
@BirdkeeperDan said:
@1p: Saying pie charts are confusing because ``They`re just not very good`` is straight contradiction and kindof rediculous.
I'm saying it's the other way around - they're not very good because they are often confusing.
@BirdkeeperDan said:
@1p: Pie charts are very commonly used
This is true, but doesn't imply anything about their effectiveness.
@BirdkeeperDan said:
@1p: with your charts I need to look at the axis to interpret, which to me means they are no better than the table which was already included.
If you remove the axis labels from both of my above charts, you can still read their message clearly: Most of Jeff's reviews are either 3 stars or 4 stars and scores on the extreme ends of the rating scale are rather rare in comparison. This distribution pattern is immediately clear when looking at the charts (especially the second one). I intentionally left out the exact data labels (eg. 39% of reviews were 4 stars), because they don't help to convey the message any better than the graph already does.
Because we're only talking about a total of five data points here: You could absorb these pretty quickly just by looking at the data table, yeah. Same goes for Marino's pie chart, because he printed the actual percentage numbers right onto it - it basically works as a fancy table. The visualization aspect is obviously lost at that point.
Wow this is amazing, love to nerd out at stats like these.
I had figured Jeff has done the most reviews but didn't realize it was as much as the other 4 combined.
@Marino: Jeff's individual chart says he has reviewed only 8 PC games. Then I saw that you said he had reviewed the most PC games, which didn't make any sense with that data. On the big comparison chart it says he actually reviewed 19.Actually, his 360 and XBLA platform numbers are messed up too. I thought you might want to know.
Anyway, I love this sort of crazy data aggregation. Good on you.
@1p : I tend to agree on the pie vs. stacked bar chart issue. Stacked bar graphs are really useful, since you can line the stacked bar for each data subset next to each other to compare them easily. For example, placing the stacked bars for Jeff and Brad next to each other, you could quickly see not only the distribution of scores for each individual, but also how they compare to one another.
And 3D charts are right out if you want to examine things in anything but the broadest terms.
@Scrawnto said:
@Marino: Jeff's individual chart says he has reviewed only 8 PC games. Then I saw that you said he had reviewed the most PC games, which didn't make any sense with that data. On the big comparison chart it says he actually reviewed 19.Actually, his 360 and XBLA platform numbers are messed up too. I thought you might want to know.
Yep. The numbers on Jeff's individual platform chart are wrong. Not sure what happened there. The big chart is correct. I'll get on fixing that.
there is nothing i like more than bizarre statistical minutiae. thank you, marino... for making us laugh about reviews... again.
"Good job." –Guile, Capcom vs. SNK 2
@BirdkeeperDan: I have to say 3D pie charts just arent as good, though bad 2D ones suck too. Why can't all charts look all high-res, colorful, and have nice shading and depth? That would be cool but then we would all have to use Numbers or something.
Basically I am saying bar graphs are the coolest.
@Marino said:
@Thompson820 said:
Really cool to see this stuff broken down, as always. I kind of want to see just how much raw data you had to work from to create this, but I fear it.
I have one gripe, and it's only because I do this kind of stuff frequently in my course, the formatting of your graphs bugs me. Every time I look at this stuff I can't help but want to order it correctly instead of having it in random order. Pie charts are meant to start with the largest chunk beginning at the 12 o'clock position and the other chunks in descending order clockwise.
That nitpicking aside, this is awesome. Do more.
As for the raw data, I could post the Excel file I guess, but it's basically just a giant table with headers for Game Title, Platform, Review Date, Review By, Staff Score, # of User Reviews, User Average. To get the data, I just had to go to each game's review page. The site already provides an average score on user reviews. The only tricky part was reviews on DLC. I had to manually pick those out and calculate an average since all DLC reviews are bundled together on the game's page.
As for the pie charts, yeah, I thought about that too. But, for whatever reason, I just stuck with a uniform 5-4-3-2-1 clockwise format. And on the giant platform pie chart, I wanted to keep it so that you could see the Sony platforms all together vs the Nintendo platforms all together, etc. I tried to keep the colors in such a way that variations of blue/purple was Sony, red was Nintendo, green/orange for Xbox/XBLA, yellow for Apple, etc.
Please do post the Excel file. Thanks!
@Lelcar said:
do we know what all the one-star reviews are?
Fable II Pub Games | XBLA | 8/18/08 | 1 | Ryan |
Rock Revolution | PS3 X360 | 10/29/08 | 1 | Jeff |
Interpol: The Trail of Dr. Chaos | XBLA | 1/9/09 | 1 | Ryan |
Master of Illusion Express: Funny Face | DSiWare | 4/7/09 | 1 | Jeff |
Velvet Assassin | X360 | 5/15/09 | 1 | Brad |
G.I. Joe: The Rise of Cobra | PS3 X360 | 8/10/09 | 1 | Jeff |
Tony Hawk RIDE | PS3 X360 | 11/20/09 | 1 | Jeff |
PowerGig: Rise of the SixString | X360 | 11/24/10 | 1 | Ryan |
Madden NFL Football | 3DS | 3/30/11 | 1 | Alex |
Lucha Fury | XBLA | 6/28/11 | 1 | Alex |
Blackwater | X360 | 12/15/11 | 1 | Alex |
South Park: Tenorman's Revenge | XBLA | 4/4/12 | 1 | Alex |
Steel Battalion: Heavy Armor | X360 | 6/19/12 | 1 | Brad |
Please Log In to post.
Log in to comment