CODEWITHSUNDEEP
node.jshttprequest
itaied
itaied

Reputation: 7107

Request a GET failed in Node

I am requesting a GET to a 3rd party api service from my node back-end.
I am getting a response of 403 forbidden:

request("http://www.giantbomb.com/api/search/?api_key=my_api_key&field_list=name,image,id&format=json&limit=1&query=street%20fighter%203&resources=game",(err,res,body) => {
    console.log(body);
})

Querying the same request in my browser return the expected results.
Any idea why this can happen?

EDIT:

Logging the response body, I receive the following page (without the JS):

<h1>Wordpress RSS Reader, Anonymous Bot or Scraper Blocked</h1>
<p>
    Sorry we do not allow WordPress plugins to scrape our site. They tend to be used maliciously to steal our content. We do not allow scraping of any kind.
    You can load our RSS feeds using any other reader but you may not download our content.
    <a href='/feeds'>Click here more information on our feeds</a>
</p>
<p>
    Or you're running a bot that does not provide a unique user agent.
    Please provide a UNIQUE user agent that describes you. Do not use a default user agent like "PHP", "Java", "Ruby", "wget", "curl" etc.
    You MUST provide a UNIQUE user agent. ...and for God's sake don't impersonate another bot like Google Bot that will for sure
    get you permanently banned.
</p>
<p>
    Or.... maybe you're running an LG Podcast player written by a 10 year old. Either way, Please stop doing that.
</p>

Upvotes: 2

Views: 1795

Answers (2)

Alongkorn
Alongkorn

Reputation: 4197

this service requires User-Agent in headers, see this example

const rp = require('request-promise')

const options = {
  method: 'GET',
  uri: 'http://www.giantbomb.com/api/search/?api_key=my_api_key&field_list=name,image,id&format=json&limit=1&query=street%20fighter%203&resources=game',
  headers: { 'User-Agent': 'test' },
  json: true
}

rp(options)
  .then(result => {
    // process result
  })
  .catch(e => {
    // handle error
  })

Upvotes: 1

kaxi1993
kaxi1993

Reputation: 4700

Include User-Agent header in request like this

var options = {
  url: 'http://www.giantbomb.com/api/search/?  api_key=my_api_key&field_list=name,image,id&format=json&limit=1&query=street%20fighter%203&resources=game',
  headers: {
    'User-Agent': 'request'
  }
};

request(options, (err,res,body) => {
    console.log(body);
})

Upvotes: 1

Related Questions