Apologies guys, we were actually pushing out new code and servers to get some of the issues fixed. We're seeing increased traffic from users (yay!) and also from bots (boo!). This is causing some places that were brittle before to fall over so it's actually on the top of our list.
As some of you know, last Thursday, we took GB and CV down for half an hour to update our redis cluster (beefier machines, replication). Yesterday, we updated our beanstalk servers to make sure they wouldn't time out with forum messages (the duplicate message bug that was around before). Unfortunately, php being php (or maybe us not being careful) it didn't take the new server's ip and decided to keep talking to the old beanstalk server. Finally fixed this morning, but not before a lot of head banging and teeth gnashing.
Gamespot is taking almost all of our resources at the minute, but we're almost there (beta planned end of this week). During this time (and even this week), we're rotating back onto GB and CV to start providing fixes. There's about a 20,000 line merge that's coming in the next couple of weeks that should ease things (for example, activities and notifications are moving to their own riak cluster which clusters better then our current setup, a new video player with better seeking and not as crash-prone, etc) and provide better stability across the board.... that and we'll chain Vinny to his seat from now on.
PM me if you have any further questions and thanks again for your patience.
Log in to comment