CODEWITHSUNDEEP
node.jsasynchronous
amagumori
amagumori

Reputation: 697

node.js async parallel function returns a single result several times

I'm trying to upload several files in parallel using node.js's async module. My code looks like: 

// fileArr is an array of objects. each object contains the attributes of the file to be uploaded - filename, path, destination path, etc.

// toUpload is an array to which i push all the functions i want to execute with async.parallel

  for (i=0; i<fileArr.length; i++) {
    var src = fileArr[i]['path']
    var dest = fileArr[i]['dest']
    var fn = function(cb) { self.upload(src, dest, headers, function(err, url) { 
      cb(null, url)   
    })  
  }
  toUpload.push(fn) 
  } // for loop ends here

async.parallel(toUpload, function(err, results) {
  console.log('results: ' + results)
})

My problem: for n = the number of functions in toUpload, the results array callback contains the result from the last parallel task in the array, n times. I can't figure this out. it seems like every function should return its own callback with (null, url) to the parallel function.

also - when i try calling the self.upload function with the definitions of src and dest directly:

self.upload(fileArr[i]['path'], fileArr[i]['dest'], headers, function(err, url) {
  cb(null, url)
})

i get a error saying "cannot read property 'path' of undefined". so, fileArr[i] is undefined. why does this happen? i feel like there is some weirdness with assignments and scope going on...

if it's not immediately obvious from the question (and the code), i'm pretty new to programming..

Upvotes: 0

Views: 1097

Answers (1)

loganfsmyth
loganfsmyth

Reputation: 161457

Keep in mind, this is essentially:

var src, dest
for (i=0; i<fileArr.length; i++) {
  src = fileArr[i]['path']
  dest = fileArr[i]['dest']
  var fn = function(cb) {
    self.upload(src, dest, headers, function(err, url) { 
      cb(null, url)   
  })  
}

which may make it clearer that by the time your fn function is called, the for loop has finished, so src, and dest will have their final loop value for every call to fn.

Similarly, for self.upload(fileArr[i]['path'], fileArr[i]['dest'], ..., by the time your function runs, the value of i === fileArr.length because the for-loop has finished.

The easiest solution for this would be to use async.map instead.

async.map(
  fileArr,
  function(file, callback){
    self.upload(file['src'], file['dest'], headers, function(err, url){
      callback(null, url);
    });
  },
  function(err, results) {
    console.log('results: ' + results)
  }
)

I'm passing null as the error because that is what you are doing in your example, but you should probably not be discarding errors since they might be important.

Upvotes: 1

Related Questions