Receiving HTML as response instead of JSON results. What can I do?

Forum Posts

Wiki Points

Followers

Reviews: 0

User Lists: 0

#1 Edited By MetalTechie

Hello everyone. I'm building a backlog tracking web app, and I was hoping I could use GiantBomb's APIs to get the required data.

Here's the thing. Instead of getting back the appropriate JSON results as a response, I'm getting some HTML that says "Wordpress RSS Reader, Anonymous Bot or Scraper Blocked" (along with more HTML). I've Googled it, and found out I had to use a User Agent. However, when I do, nothing is changing.

My code is in Python and I'm using Unirest to send the request.

I'm unable to receive the correct response. What can I do to fix this?

7 years ago

thisiseric

Forum Posts

Wiki Points

Followers

Reviews: 0

User Lists: 0

#2 Edited By thisiseric

That response means you are still getting blocked by the user agent filter. Maybe you're not entering it correctly? The request should look something like this.

response = unirest.get('http://www.giantbomb.com/api/games/?format=json&api_key=1234abcd', headers={'User-Agent': 'metaltechie app'})

I use requests, personally, but the idea with unirest is exactly the same.

7 years ago

mayhew3

Forum Posts

Wiki Points

Followers

Reviews: 0

User Lists: 0

#3 mayhew3

I'm getting back an HTML page that reads:

<body>Redirecting to <a href="https://www.giantbomb.com/api/game/3030-46631/?api_key=XXXXXX">

This only started happening recently, within the last month, and had been working successfully for a couple years. I'm using 'User-Agent': 'Mozilla/4.0'. Any ideas on what may have changed, or how to fix it?

Thanks,

Mayhew

7 years ago

mikewhy

595

Forum Posts

Wiki Points

Followers

Reviews: 0

User Lists: 0

#4 mikewhy

Are you following HTTP redirects? There was an announcement about preparing for HTTPS a while ago. When enforcing HTTPS, redirecting HTTP to HTTPS is what you do.

7 years ago

mayhew3

Forum Posts

Wiki Points

Followers

Reviews: 0

User Lists: 0

#5 Edited By mayhew3

Thanks. I hadn't seen the https announcement. Changing to using https did the trick!

7 years ago

Move topic to another board