#1 Edited by andy (427 posts) -

Just deployed several changes.

  • Added a "Search" resource.  This can be used to search for things by name on Giant Bomb.  Each result from the search resource will will look identical to how it would be represented if it was queried from its corresponding list resource, except it will have one additional field called "resoruce_type", which indicates the name of the resource.  For example, if a result from the Search result was the Metal Gear Solid 4 game, it would have all the same fields and data as if it were queried from the /games/ resource, with the addition of the resource_type field having a value of "game".  Take a look at the documentation for filtering by type.
  • Fixed some documentation errors on Releases.
  • Added a "field_list" filter on all resources.  Using this filter, you can retrieve only the fields you want.  This will reduce the size of the response, and in some cases, also reduce the time required to process your request on the server.
  • Fixed a couple misc. bugs with negative offsets.  Thanks to jlrm01 for the bug report.

I nearly have the work done to export the site reviews and ratings.  I'd expect those to come out tomorrow.  As always, please let me know if you see any problems.
Staff
#2 Posted by LordAndrew (14426 posts) -

Looks good, at least from the documentation. I haven't actually tried it out yet.

#3 Edited by jlrm01 (1992 posts) -

In my tests so far it seems to be working....

When given an incorrect resource type, it gives an Internal Server Error, for example:
http://api.giantbomb.com/search/?query=sumaga&format=json&api_key=<Your API key goes here>&resources=games&offset=0

Maybe it should give a regular error instead?

Also, I found that if you use a space after the comma, only the first type applies:
http://api.giantbomb.com/search/?query=SAMUS&api_key=<Your API key goes here>&resources=character%2C+game&format=json

I guess this is just developer error, and once bitten won't happen again (I always put a space after a comma, so I fell for it the first time). ^_^

#4 Posted by andy (427 posts) -
jlrm01 said:
"In my tests so far it seems to be working....

When given an incorrect resource type, it gives an Internal Server Error, for example:
http://api.giantbomb.com/search/?query=sumaga&format=json&api_key=<Your API key goes here>&resources=games&offset=0

Maybe it should give a regular error instead?

Agreed.  Just fixed this.
Staff
#5 Posted by jlrm01 (1992 posts) -

I get the error that you added for objects, but objects are listed as available in the docs. For example:

http://api.giantbomb.com/search/?query=sword&api_key=<Your API key goes here>&limit=1&resources=object&format=json

Gives an error.

#6 Posted by andy (427 posts) -

Yup, you're right.  Gimmie a few mins.

Staff
#7 Edited by jlrm01 (1992 posts) -

This one is old, I guess, but giving a negative offset (or a really big offset) gives a 500 error. Negative limits don't seem to cause an error.

#8 Posted by andy (427 posts) -
jlrm01 said:
"This one is old, I guess, but giving a negative offset (or a really big offset) gives a 500 error. Negative limits don't seem to cause an error."

Fixed this, as well as the "object" bug you mentioned above.  I also took the time to add the "field_list" filter on all resources so you can limit what fields are coming back in the response.  I updated the first message in this thread with the updated changes.

Thanks for the testing!

Staff
#9 Edited by jlrm01 (1992 posts) -

Haven't tested the new changes yet (will soon), but I have a few questions....

Will regular expressions be added for the search? So I could do /search/?regexp = 's.*s' or something like that, for the names? I know other filters were mentioned in other threads, but if you include regular expressions one can do a lot....

Can I do web scraping without getting banned? ^_^; For example, for testing purposes, I wanted to create a script that gets all possible fields for all possible resource types, and tests them all or most (with a limit of 1 or something small, to avoid wasting too much bandwidth). I don't want to type that in myself, but I could easily (probably, haven't checked the structure of the generated HTML) get the info off the docs. Since there are no introspection facilities, I can get the available resources / filters and such this way, too. (There is a much easier way, I guess, for the fields... now that I think about it). Should I?

Also, users are never given as results, but it is useful to me to know, for example, which images in certain  pages were not uploaded by me and get their thumbnails (this is all I need), to avoid uploading duplicates (also, caching all images locally, such as to never request an image more than once). I'm doing this manually right now (with a Firefox extension, actually, page by page, but getting all thumbnails), but for the program I'm making (non-web based, non-iPhone... just a plain old Python + wxPython regular app) it would be really useful, I think. (It could be used to search for duplicates in really huge pages, too, eventually, once I adapt the "mrquery" script in my program, instead of using another app). I will not store user info for anything. And I suspect web scraping might be useful in other circumstances, where the API won't give me a certain info I need.

Can I do this? Or, I mean, should I? Sorry about the long post... I've always wanted to ask.... ^_^

#10 Edited by jlrm01 (1992 posts) -

(I decided not to edit my previous post and and put this there... that post is too long already.)

I tried the offsets... but... very big numbers give a Enter a reasonable offset value message, but negative numbers get treated as if the offset was the default. Shouldn't it give the same error as for huge numbers? After all, such an offset value would usually be the result of an error in the program, and an error message would be more useful, I think. ^_^

#11 Posted by snide (2413 posts) -

Hey jirm01 -

We've omitted the image attachments on resources (outside of the default image) on purpose. Although we're pretty open with our database, as a business it didn't make sense to give everything away. It's also the one thing that would be fairly hard for us to track usage of, not to mention that images and artwork sits in a grey area as far as copyright goes.

As to user data (whether it be attached to images or not), we don't support that info because of privacy concerns. I imagine in the future we'll figure out a way to make such data available, but it's not something we want to rush into without considering the consequences.

Perhaps if you give us an idea of your project we can help figure out a safe solution. I'll let Andy address your actual bugs.
#12 Edited by ThaNerd (10 posts) -

Absolutely great! it gives exactly what i needed to implement it on my website... Now just remains to decide wether cacheing data on my server or not. I'll dig into the terms of use to see if i can. But i guess reducing greatly the bandwidth use is always a good option for API providers :)

I might even try putting a php front-end class together  for this api...

#13 Posted by TheBeast (1931 posts) -

Great work! The search functionality is excellent!

I can't help but feel that the individual list resources seem a bit useless compared to the all-powerful search - but at least the option is open to people.

I haven't had a chance to test it fully as of yet, but I'll provide some real feedback when I can.
#14 Posted by doogiemac (22 posts) -

This is awesome, you guys.  Can't wait to implement the field list filtering!

#15 Posted by andy (427 posts) -
TheBeast said:
"Great work! The search functionality is excellent!
I can't help but feel that the individual list resources seem a bit useless compared to the all-powerful search - but at least the option is open to people.

I haven't had a chance to test it fully as of yet, but I'll provide some real feedback when I can.
"

The real use of list resources comes into play if you want to create a DB locally and use our content to completely power your site.  Also, being able to filter on resource-specific things, like game ratings or whatever -- of course that means I need to actually add the filters to the resources ;)

Staff