CODEWITHSUNDEEP
javascriptnode.jsecmascript-2016
Ollie2619
Ollie2619

Reputation: 1295

Keep getting socket hang up when using node-fetch

I am trying to fetch a url to get the headers and make sure it returns the content type, so that I can check to make sure it contains "image"

So far I am just looping through a database to get the image urls to pass into fetch to get the content type.

It gets so far and then gets a socket hang up error.

here is the function I am calling

const getImage = async (url) => {
  try {
    const res = await fetch(encodeURI(url));
    console.log(res.headers.get("content-type"));
  } catch (err) {
    console.log(err);
  }
};

however after so long it starts erroring and node-fetch produces the error below:

FetchError: request to http://www.fakeurl/image.jpg failed, reason: socket hang up
at ClientRequest.<anonymous> (/home/user/code/url-check/node_modules/node-fetch/lib/index.js:1455:11)
    at ClientRequest.emit (events.js:321:20)
    at Socket.socketOnEnd (_http_client.js:463:9)
    at Socket.emit (events.js:333:22)
    at endReadableNT (_stream_readable.js:1201:12)
    at processTicksAndRejections (internal/process/task_queues.js:84:21) {

Im not sure what I am doing wrong, im guessing the server is getting too many requests at once, but dont know how to slow it down or queue it up to try again

edit

here is the database loop that calls the function above:

connection.query(sql, (error, results, fields) => {
  if (error) {
    return console.error(error.message);
  }

  for (let i = 0; i < results.length; i++) {
    getImage(results[i].image_url);
  }
});

connection.end();

Upvotes: 6

Views: 4481

Answers (1)

Ziad Ali
Ziad Ali

Reputation: 56

There are a couple of reasons this may be happening:

  • The server is still processing the request prior
  • You did not close the port after each call

What I think you should do:

  • Close the port after each request is satisfied from the DB.
  • Search if your DB provider has an option to getBatch(), which you send everything you want to the DB at once, and it sends them to you in one response.

Upvotes: 1

Related Questions