API responses to be limited to 100 returned items

In the near future, we will be enforcing limits in our API on the number of items that can be requested in a single call to 100 items. In the past, we had some API users exceeding this amount which caused headaches for our developers, so we are taking this step to make sure the site and the API are up and available to everyone as close to 24/7 as possible.

If you have concerns or questions, feel free to post them here.

Sounds like a good idea. But the last updates seems to be only about limitations. Will there be any new features any time soon?

Will be there a cursor or some other means to continue a query with a second request when you want 101+ results?

@ascagnel: The 'offset' parameter is what you're looking for. I actually thought API calls were already limited to 100 results per call, but I guess must be mistaken.

@tjockapa: We are getting scraped, and abused by very many users. What are indeed limitations are only being done so we don't have to discontinue offering the API at all. It's really expensive to offer this type of access as a public feature, so we are in real danger if we don't protect adequately. I'm a cheerleader for the API, and never want it to go away.

Also, by making these changes, it will give us a better chance to make meaningful changes to the API, including offering more/new data, new filters, etc. But we can't do that until we stop the attacks first.

@chaser324: Some were limited but it was more specific to certain queries than others. We are just doing some cleaning up certain queries that return too many results, based on complexity of joined data due to associations, AKA we have a lot of crazy data.

Seems very reasonable.

Thank you for the update. Hope you guys can move on from protecting the API to adding new features soon.

@jslack I have full understanding for that, and as i said; it sounds like a good idea. But could we maybe have some sort of a roadmap on upcoming features (including "limitations")? As looking at the API now, including this forum, it's not a lot of activity. And by letting us know what will come maybe you can get valuable input before actual development.

@clidus: @tjockapa:Absolutely. Something I'm working on now is updating some of the documentation reflecting policies, so our API users have a better understanding of guidelines and changes. And yes, a roadmap is something I've been working on creating. I think Jeff and Tony are open to new ideas for the API, but they wanted to prioritize stability and fixes first. But yes, a roadmap is being discussed. The API / wiki are things I have a lot of personal interest in, and I'll do what I can to get a roadmap out there.

Thanks for the hard work @jslack!

@rorie: If one intends to simply get the game list and id for - let's say - windows games (20006 games!)

http://www.giantbomb.com/api/games/?api_key=apikey&field_list=name,id&filter=platforms:94&offset=0

there would be a total of 201 requests...correct?

Wouldn't it trigger the "Rate limit exceeded. Slow down cowboy" error !?

By the way, it's still not clear to me: what's the rate limiting? When it occurs, how much time before user gets access again?

Why is it so difficult to get a reply on such basic questions?

Perhaps am I simply mistaken about the finality of the API !? Is it to be used occasionally or on a few items only? (no offense, just willing to understand!!)

Simply said: is it conceivable to use this api to retrieve info for thousands games in an application?

@jayme65: If your app is doing something on a scale where you're concerned about hitting the rate limit, you need to be caching API data in your own database so that you reduce the number of necessary queries.

@chaser324: Thanks for your reply! So you confirm me that this is the official (and permitted) practice: grabbing info from GB db and then serving it to my app users?

Thank you!

@jayme65: As long as you're properly attributing Giant Bomb as the source of the data, that's the way you should do it.