Avatar image for metaltechie
#1 Edited by MetalTechie (2 posts) -

Hello everyone. I'm building a backlog tracking web app, and I was hoping I could use GiantBomb's APIs to get the required data.

Here's the thing. Instead of getting back the appropriate JSON results as a response, I'm getting some HTML that says "Wordpress RSS Reader, Anonymous Bot or Scraper Blocked" (along with more HTML). I've Googled it, and found out I had to use a User Agent. However, when I do, nothing is changing.

My code is in Python and I'm using Unirest to send the request.

I'm unable to receive the correct response. What can I do to fix this?

Avatar image for thisiseric
#2 Edited by thisiseric (26 posts) -

That response means you are still getting blocked by the user agent filter. Maybe you're not entering it correctly? The request should look something like this.

response = unirest.get('http://www.giantbomb.com/api/games/?format=json&api_key=1234abcd', headers={'User-Agent': 'metaltechie app'})

I use requests, personally, but the idea with unirest is exactly the same.

Avatar image for mayhew3
#3 Posted by mayhew3 (2 posts) -

I'm getting back an HTML page that reads:

<body>Redirecting to <a href="https://www.giantbomb.com/api/game/3030-46631/?api_key=XXXXXX">

This only started happening recently, within the last month, and had been working successfully for a couple years. I'm using 'User-Agent': 'Mozilla/4.0'. Any ideas on what may have changed, or how to fix it?

Thanks,

Mayhew

Avatar image for mikewhy
#4 Posted by mikewhy (88 posts) -

Are you following HTTP redirects? There was an announcement about preparing for HTTPS a while ago. When enforcing HTTPS, redirecting HTTP to HTTPS is what you do.

Avatar image for mayhew3
#5 Edited by mayhew3 (2 posts) -

Thanks. I hadn't seen the https announcement. Changing to using https did the trick!