CODEWITHSUNDEEP
node.jsnode-request
王仁宏
王仁宏

Reputation: 396

There is no body content when nodejs-request get but work in browser or curl

request get no body content with header "content-length: 0".

resource: https://www.doe.gov.taipei/OpenData.aspx?SN=8A3B3293C269E096

It is a rss link, work well in browser and curl. curl test on linux, and wget works too.

here is simple code:

const req = require('request');
req.get({
  url: 'https://www.doe.gov.taipei/OpenData.aspx?SN=8A3B3293C269E096',
  rejectUnauthorized: false
}, (err, res, body) => {
  console.log(res)
});

test it online simulator get same result https://repl.it/repls/ScalyAquamarineObjectpool

i think it should be a text/xml content, not empty.

response:

{
statusCode: 200,
  body: '',
  headers:
   { 'cache-control': 'private,No-cache',
     'set-cookie':
      [ 'ASP.NET_SessionId=kzdhyxq2grttp5awwpqzxyen; path=/; secure; HttpOnly' ],
     'x-frame-options': 'SAMEORIGIN',
     'strict-transport-security': 'max-age=0',
     'x-xss-protection': '1; mode=block',
     'x-content-type-options': 'nosniff',
     'content-security-policy':
      'frame-ancestors \'self\' https://www-mgr.gov.taipei http://www-mgr.gov.taipei',
     date: 'Wed, 03 Apr 2019 06:44:47 GMT',
     connection: 'close',
     'content-length': '0' },
  request:
   { uri:
      Url {
        protocol: 'https:',
        slashes: true,
        auth: null,
        host: 'www.doe.gov.taipei',
        port: 443,
        hostname: 'www.doe.gov.taipei',
        hash: null,
        search: '?SN=8A3B3293C269E096',
        query: 'SN=8A3B3293C269E096',
        pathname: '/OpenData.aspx',
        path: '/OpenData.aspx?SN=8A3B3293C269E096',
        href:
         'https://www.doe.gov.taipei/OpenData.aspx?SN=8A3B3293C269E096' },
     method: 'GET',
     headers: {} }
}

Upvotes: 1

Views: 985

Answers (2)

user826955
user826955

Reputation: 3206

To your updated question: It seems like adding a user-agent does the trick:

const req = require('request');
req.get({
  url: 'https://www.doe.gov.taipei/OpenData.aspx?SN=8A3B3293C269E096',
  rejectUnauthorized: false,
  headers: {
    'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/72.0.3750.0 Iron Safari/537.36'
  }
}, (err, res, body) => {
  console.log(body)
});

Upvotes: 0

user826955
user826955

Reputation: 3206

The signature of your callback function is wrong. In Node.JS, usually the first parameter to the callback is the (optional) error object.

Try this:

const req = require('request');
req.get({
  url: 'https://www.doe.gov.taipei/OpenData.aspx?SN=8A3B3293C269E096',
  rejectUnauthorized: false
}, (err, res, body) => {
  console.log(res)
});

Upvotes: 1

Related Questions