CODEWITHSUNDEEP
node.jsasynchronous
m_vdbeek
m_vdbeek

Reputation: 3774

Multiple async http get requests

I'm struggling with asynchronous events in Node.js.

My app is doing a first HTTP GET request and then based on some information found in the body of the response it is looping a certain amount of times.

At each iteration it calls a function which sends other HTTP GET requests and returns a result pushed in the elements array.

My problem is that I want to console.log() the elements array when all the http requests are done. Here's my code :

http.get('mysite.com', function(res) {
    var body = '';
    res.on('data', function(chunk) {
        body += chunk;
    });
    res.on('end', function() {
        var last = 5,
            elements = [];
        for(var i=0; i < last; i++) {
            elements.push(myRequest(i));
        }
        console.log(elements);
        // Do some stuff with the elements array after it has been populated
    });
}).on('error', function(error) {
    console.log(error);
});

var myRequest = function(i) {
    http.get('mysite.com/page' + i, function(res) {
        var body = '';
        res.on('data', function(chunk) {
            body += chunk;
        });
        res.on('end', function() {
            // Do parsing here
            return parsed;
        });
    }).on('error', function(error) {
        console.log(error);
    });
};

I thought about using the async module but I'm not sure how to use it in this use case.

Thanks in advance !

Upvotes: 0

Views: 2032

Answers (1)

mindandmedia
mindandmedia

Reputation: 6825

I would use the async library and maybe use a queue. but if you want to do it by hand, without having tested anything below and not really sure it's the cleanest way, but it might give you ideas.

pass a callback to your request-function:

var myRequest = function(i, cb){
   ...
   res.on('end', function(){
      cb(parsed);
   });
}

and above onEnd-call would look more like this:

var last = 5, elements = [];

var rep = function(i){
  if( i < last ){
     myRequest(i, function( result ){
        elements.push(result);              
        rep(i+1);
     });
  } else {
    console.log(elements);
    // Do some stuff with the elements array after it has been populated
  }
}

res.on('end', function() {
   rep(0);
});

using async would come out a lot nicer looking. gotta run now though...

Upvotes: 2

Related Questions