Next Class Quest Stats

Giant Bomb's first Sponsored-class quest is on, courtesy of Best Buy, and the community is facing a decent challenge for a simple quest set! Naturally, I'm doing my usual thing of tracking progress and stats. If you want to talk about the merits of sponsored quests, there's another thread for that. If you're looking for graphs and numbers, that's what this post is here for. 
 
Unfortunately, for one reason or another (likely the requirement to have five followed users on at once or a saturation in willing quest participants), we've failed. With an expected end time of Sunday and an actual end time of Friday, I poured a little too much hope into our initial chances.
 
Here's the final graph of progress.

Final Progress for Next Class
The gold bar at the top was our goal of 6,000 user completions. The red line's the general pace we needed to maintain to have a chance at success. The green line is the progress we made over the course of the quest.
 
Final stats:
  • We needed 6,000 completions, but only attained 4,009.
  • We had an overall average of 36.5 completions per hour. Given our shorter timeframe, we needed an average of 54.67 completions per hour.
  • To complete this quest given our final rate of completion, we would have needed at least 164.26 hours, or until Monday.
  • Based on rough site statistics, 6,759 people have completed one of the quests in this set but haven't finished them all. This means 37% of the people who finished one of the quests managed to finish the whole set. On average, the incomplete users have 2.2 out of five quests complete.
  • My current Arbitrary Estimation of Success should have hit 0% a lot sooner*.
  • All the Whiskey sites missed their quest goal, but Anime Vice came the closest, even though only 200 people had to complete their quest.
 
Here's a comparison vs. previous quests tracked:
Quest Comparison Graph as of Next Class

As always, if you have any requests, questions, or criticisms of my stat tracking, don't hesitate to ask! 
 
*Arbitrary Estimation of Success is an arbitrary number based on a faulty formula, and is subject to change drastically in the event of wild, unexpected shifts in user completion.
94 Comments
95 Comments
  • 95 results
  • 1
  • 2
Edited by PsEG

Giant Bomb's first Sponsored-class quest is on, courtesy of Best Buy, and the community is facing a decent challenge for a simple quest set! Naturally, I'm doing my usual thing of tracking progress and stats. If you want to talk about the merits of sponsored quests, there's another thread for that. If you're looking for graphs and numbers, that's what this post is here for. 
 
Unfortunately, for one reason or another (likely the requirement to have five followed users on at once or a saturation in willing quest participants), we've failed. With an expected end time of Sunday and an actual end time of Friday, I poured a little too much hope into our initial chances.
 
Here's the final graph of progress.

Final Progress for Next Class
The gold bar at the top was our goal of 6,000 user completions. The red line's the general pace we needed to maintain to have a chance at success. The green line is the progress we made over the course of the quest.
 
Final stats:
  • We needed 6,000 completions, but only attained 4,009.
  • We had an overall average of 36.5 completions per hour. Given our shorter timeframe, we needed an average of 54.67 completions per hour.
  • To complete this quest given our final rate of completion, we would have needed at least 164.26 hours, or until Monday.
  • Based on rough site statistics, 6,759 people have completed one of the quests in this set but haven't finished them all. This means 37% of the people who finished one of the quests managed to finish the whole set. On average, the incomplete users have 2.2 out of five quests complete.
  • My current Arbitrary Estimation of Success should have hit 0% a lot sooner*.
  • All the Whiskey sites missed their quest goal, but Anime Vice came the closest, even though only 200 people had to complete their quest.
 
Here's a comparison vs. previous quests tracked:
Quest Comparison Graph as of Next Class

As always, if you have any requests, questions, or criticisms of my stat tracking, don't hesitate to ask! 
 
*Arbitrary Estimation of Success is an arbitrary number based on a faulty formula, and is subject to change drastically in the event of wild, unexpected shifts in user completion.
Moderator
Posted by James_ex_machina

This quest is difficult.

Posted by Wrect
@James_ex_machina:
Indeed.  Difficult and enigmatic... and extemporaneous.
Posted by Rehehelly

I'm such a stats nerd.
 
I think this quest will be done in the next 2.5 days.

Posted by BraveToaster

I bet next week will be another Trivia quest.

Posted by Video_Game_King
@Rehehelly said:
" I'm such a stats nerd."
Same here. In fact, I'm curious as to the trends one would see in my game ratings.
Posted by RecSpec

I'm worried about this one.
 
It seems easy enough, but 6000 is a pretty hefty number.

Posted by gpbmike

These graphs always start off so well.

Posted by ch13696

Wait. This quest set is hard? But how? You comment on something, click a link, have some friends online with you, and I forgot what the last one was. Either way, they were pretty damn easy.

Posted by fulcilivez

niiiiice

Posted by PsEG

I do think this quest will be rather interesting to watch over time. It's exhibiting a trend very similar to that of X-Treme and Hell, It's About Time, outpacing both to reach the 2,000 mark while curving about the same. However, both those quests stop at 2,000, so there's not much data to tell us how they would have reacted over a longer period of time. This will give us a first glimpse at two fronts: How a quest's progression is affected by a more daunting goal, and how a quest with a high completion rate is affected over the span of at least one full 24-hour cycle.
 
This quest has indeed slowed down significantly in the night, more so than the Starcraft II quest did. I think part of this is that the Starcraft II quest was nearing its end in the middle of the night, so there was added fervor to complete it for bonus XP before it was too late. I also think Next Class is much harder to complete late at night because of the quest to be online at the same time as five followed users. Traffic is almost certainly lower now, so it's not a great time to catch an abundance of people online at once.
 
As I noted earlier, the key will be whether or not this quest regains its steam during the day tomorrow. I think 6,000 users for a target is just about perfect for extending this quest-based drama well into the week.

Moderator
Posted by x52z

delicious!

Posted by Dynoro

Wasn't the Limited Edition Halo Reach quest set sponsored too (and featured on Halo Waypoint)?

Posted by mracoon

What's the formula you use to work out the Arbitrary Estimation of Success?

Moderator Online
Posted by RVonE

I don't think we'll make it.
Online
Posted by CitizenKane
@RVonE said:
" I don't think we'll make it. "
This quest is by far the easiest we have ever had and we still have all week to get the next 2/3 out of the way.
 
I think we'll make it.
Posted by RVonE
@CitizenKane said:
" @RVonE said:
" I don't think we'll make it. "
This quest is by far the easiest we have ever had and we still have all week to get the next 2/3 out of the way.  I think we'll make it. "

I like your optimism. I know it's a really easy quest, but the 6000 number is really high. People who are going to get it, are likely to get it early. I hope you're right, though.
Online
Posted by PsEG
@Dynoro said:
" Wasn't the Limited Edition Halo Reach quest set sponsored too (and featured on Halo Waypoint)? "
It might have been sponsored, yeah, and the Ask Me Anything and Halo 2 Xboxalypse montage videos were made available on Waypoint. I think it was classified in the system as limited edition, though, rather than the fancy sponsored classifcation Best Buy has for Next Class. Just semantics on my part, really.
 
@mracoon said:
" What's the formula you use to work out the Arbitrary Estimation of Success? "

Short answer: It's an abuse of the ZTEST function in Excel.
 
Long answer: With the Arbitrary Estimation of Success, I'm hoping to use my periodic readings of data to calculate where the goal amount of completions would fall on a normal distribution of estimated totals, giving us the odds of having a performance better than 6,000.
 
Every time I take a reading of the number of completions at a certain time, the spreadsheet I use to record this data calculates it against the previous reading to determine a short-term rate of completion, and then uses that rate to estimate how many completions we would achieve by the end of the quest (this is also calculated long-term using the running totals, but that's irrelevant to this example). For AES, I currently calculate it three times, using the last five, ten, and twenty-five short-term estimates. I used to take all the readings into account, but that made the AES number very resistant when a challenge had a strong start, and it'd take way too long for it to move away from 100%.
 
From there, I basically use a function in Excel, which I'm hoping is determining where exactly the desired amount of completions would fall on a normal distribution. The formula is basically this in Excel:
 
1 - ZTEST( [last 5, 10, or 25 readings], [quest goal] ) * 100
 
With the readings serving as the data array and the quest goal of 6,000 as the number to test against them. I leave the field for sigma blank since there's really no population to grab a standard deviation from, so the standard deviation of my sample is used instead. From the three calculations returned to me, I usually average the three calculations together to get a final AES, depending on the trend of the data and if one AES calculation is too high or low.
 
I'm aware there's almost certainly heavy flaws in this calculation, which is part of why I'm labeling it as arbitrary. I unfortunately don't have any statistics books lying around, and my memory of the basic stats classes I've taken is foggy at best. If anyone with a good knowledge of stats wants to rip this apart and offer a better way to estimate the odds of success, I'm more than willing to listen.
Moderator
Posted by TheDrake

I really hope that there are enough people willing to participate in this. Its super easy and can be done in less than 10min. If anyone needs more people to follow for the quest I leave giantbomb up all day, of course a lot of the staff are online most of the day as well.

Posted by Azteck

I'm not sure if we'll make it this time either. Most likely, the people who care about quests in any way have already gotten this one, and the stragglers will pile up in the coming week. If there are 4000 of them though.. Hard to say.

Posted by Skeetz

Advertising on the internet in on inoffensive way is hard to do, but  think this is one good way.

Posted by mosespippy

I don't think we are going to get it. How many quest sets have over 6000 completions so far? I don't think there are 6000 quest hunters. I'd say it's closer to 2000 or 3000. We need the slack to be picked up by casuals.

Posted by icytower38

I think we will finish someplace around 4,000. Easiest quest or not, I don't think there are enough people out there that care enough to do it to get us to 6,000.

Posted by RobbieMac

 The quest has really begun to stall.  I finished over 16 hours ago and since that point only 540 more users have completed the quest.

Posted by Dynoro

I can't see us making the 6000 - looks like topping out at 4000 at most atm

Posted by Jaserno

Everyone make an extra user account and do the quest, then we'll get it.

Posted by CitizenKane
@Jaserno said:
" Everyone make an extra user account and do the quest, then we'll get it. "
No.  Just no.
Posted by Jaserno
@CitizenKane said:
" @Jaserno said:
" Everyone make an extra user account and do the quest, then we'll get it. "
No.  Just no. "
That would get it done. But it would also be impeding the regulations of the site that say you are not allowed more than one user account? Amirite?
Edited by CitizenKane
@Jaserno said:

" @CitizenKane said:

" @Jaserno said:
" Everyone make an extra user account and do the quest, then we'll get it. "
No.  Just no. "
That would get it done. But it would also be impeding the regulations of the site that say you are not allowed more than one user account? Amirite? "
Yes.  Multiple accounts is against the rules here at Giant Bomb, no matter what circumstance.
Posted by TMThomsen

I foresee this quest will not be completed.

Posted by FluxWaveZ
@mosespippy said:
" I don't think we are going to get it. How many quest sets have over 6000 completions so far? I don't think there are 6000 quest hunters. I'd say it's closer to 2000 or 3000. We need the slack to be picked up by casuals. "
This.  I find it highly unlikely that 6,000 people will complete the quest.  It won't get completed.
Posted by Cyphalon

Roar   zygomata 

Edited by wrathofconn
@PsEG: Thanks for doing this again, these threads are good stuff.
 
I think it's possible that this will get completed, but the graph is going to be interesting either way. I think that during the week there will be lulls during the day rather than at night like on the weekend, should be an interesting curve. The only part that will slow us down is Late Night Cram, I'm sure way more than 6,000 people will complete all the other parts accidentally this week even if they aren't into quests.
Posted by Martin

Shouldn't be difficult to complete.
Posted by Lies

These things are frontloaded to the max, doubt we'll reach the goal.

Posted by one_2nd
@CitizenKane said:
" @Jaserno said:
" Everyone make an extra user account and do the quest, then we'll get it. "
No.  Just no. "
lmao. good idea, bro!
Posted by RecSpec

The only quests that seem to really push the community are the Conquests, because there are actual prizes on the line.
 
These always start with a bang, then just die off.

Posted by PsEG
@RecSpec said:
" The only quests that seem to really push the community are the Conquests, because there are actual prizes on the line.  These always start with a bang, then just die off. "
I think the only quest sets we'll ever see that won't start off with a bang are difficult quests that take time to complete -- Scavenger quests with difficult answers are likely to fit this bill. I wish I had recorded the first 47 hours of the Lovin' Cup quest, because I'm pretty sure it started off very slowly until the hints made it very obvious for anyone who truly wanted it. Regardless, yes, there's always going to be an attempt at an initial surge. People either want to be #1 or get the nice and shiny new quests completed for fun. It's like MMOs after major patches.
 
Back to my main point. I've updated the graph for today, and as you can see, we did indeed slow down a bit. However, we are still on pace. Barring a major unforeseen slowdown in the near future, this should be close. That's the sign of a good target, I think. I've also changed the way I'm going to post short-term stats to be a little more accurate and less arbitrary, and thanks to some nifty stats on the site I hadn't seen before, I can point out a few new tidbits of information. 
 
AS OF AUGUST 4TH, 12:01 AM EDT:
  • We're 34.8 hours in, with an estimated 123 hours to go.
  • 3,170 users have completed this quest. 2,830 more need to complete the set if we want bonus XP.
  • Our long-term pace is now 91.2 completions per hour. At that pace, we'll complete the set in 65.8 hours, and could potentially have 14,384 completions given the whole week (that is an illegal number of completions at this point, though).
  • When we started, we needed more than 38.03 completions per hour to succeed. Now we only need more than 23.01 completions per hour.
  • Over the last 24 hours, 774 people completed this quest set.
  • The completion rate for yesterday was 32.25 completions per hour. At that rate, we'd complete this quest in 122.55 hours, and given the full week at that rate, we could potentially amass 7,137 completions.
  • Based on site statistics, 5,185 people have completed one quest in this set, but have NOT completed the entire set.
  • On average, people who haven't completed the quest set have about 2 of the 5 quests finished.
 
I'm apprehensive, but today's numbers are more promising than what I expected. Hopefully the trend stays relatively stable.
Moderator
Posted by one_2nd

We got this, boyz. 

Posted by BunkerBuster

The graph has the uncanny power to change my emotion at the end of the day. For now I'm mildly enthused but I can't wait to start being depressed after Wednesday.

Posted by D_O_A

No way enough people are going to finish this quest in time. 
 
Oh and congratulations on finally finding a non intrusive way to serve advertisements 

Posted by AZChristopher

More than half way there. Wonder if we get a boost going into the weekend to just barely get this one.

Posted by AZChristopher

I just saw the following link posted in another thread.
 
 http://www.giantbomb.com/quest/activity/
 
On the right side it shows stats for the current quest.  8839 have completed at least one step of the quest. With as easy as this one is, we'll see if they can finish all the steps.

Posted by wrathofconn
@AZChristopher: I pointed this out earlier in the thread, but I think looking at the number of users who have completed one or two parts is deceiving, since all but the link-clicking quest are amazingly easy to complete by accident. So if those people were trying to complete the set I am sure we'd get to 6,000 but I don't think that's the case.
Posted by Alphawolfy

I think this one could go either way. 6000 is a lot of people, but the quest is also quite easy. More than halfway there! If you're reading this and have not finished the set... please do :P

Posted by Tordah

My god, this topic is like the Inception of nerdiness. We're all on a website about video games (level 1), we do pointless quests that mean absolutely nothing (level 2), and now we have an in-depth analysis of said quest (level 3). Mind. Blown.
 
I wonder what Limbo is like on Giant Bomb?

Posted by cebrigham

Questing on this site has become my new favorite social networking game. I don't mind sponsored quests, and it's fun to check out the progress with fancy charts. Doing my part!

Posted by AdvanceStrat

I wouldn't count it out yet.  There's bound to be plenty of people that care about the quests, that just can't get on every day.   Myself for example.  That'll surely boost the numbers a bit as the week goes on.
Posted by PsEG

Another day, and our pace has slowed down again, just slightly below where we needed to be. It's not the death knell yet, but if the pace drops significantly tomorrow, we might drop below the red line on Friday. Otherwise, this will go into the weekend, where we'll either get a surge from weekend warriors or fall frustratingly short. Have I mentioned that I think this is the perfect target number? I think I've said that a few times.
 
The graph's been updated, and here's some daily stats:
 
AS OF AUGUST 5TH, 12:02 AM EDT:

  • The quest is 58.8 hours in, with an estimated 99 hours to go. I say estimated because "end of the week" isn't very specific. I'm banking on that being the end of Sunday, though.
  • 3,630 users have completed this quest. 2,370 more need to complete the set for that huge helping of bonus XP. I'm also really surprised that this number's been so round and easily divisible by ten for the last few days.
  • Our long-term pace dropped to 61.8 completions per hour, which isn't too surprising considering this number was heavily inflated by the first day surge. Regardless, at this pace, we'll complete the set in 97.2 hours, and could potentially have 9,741 completions given the whole week (again, heavily inflated due to day one's returns).
  • When we started, we needed more than 38.03 completions per hour to succeed. Now we need more than 23.95 completions per hour, up .94 from yesterday.
  • Over the last 24 hours, 460 people completed this quest set. To keep pace and prevent any worries, 552 users would have been nice, so we essentially fell short by 92 users.
  • The completion rate for yesterday was 19.17 completions per hour. At that rate, we'd fail this quest narrowly, with 5,528 completions.
  • Based on site statistics, 5,785 people have completed at least one quest in this set, but have NOT completed the entire set.
  • On average, people who haven't completed the quest set have 2.2 of the 5 quests finished.
 
I know I'm optimistic to a fault, but this isn't the end of our chances. It's a mere setback with a weak morning and a short afternoon of gains, and a solid tomorrow or weekend showing would keep our chances very much alive. Barring a huge flatline in the next few days, this is going to be a quest where every person counts, and yeah, I'm a stuck record saying that, but that's the way it should be, damn it.
Moderator
Posted by ajamafalous
@PsEG said:
" Based on site statistics, 5,785 people have completed one of the quests in this set but haven't finished them all. On average, these users have 2.2 out of five quests complete. "
Super surprised by this, considering this quest set took me all of one minute to complete.
  • 95 results
  • 1
  • 2