CODEWITHSUNDEEP
javascriptnode.jsasynchronous
Kirk Ouimet
Kirk Ouimet

Reputation: 28404

Need Help Thinking How to Program Asynchronously

I'm using NodeJS to walk over a list of files and generate an MD5 hash for each one. Here's how I would normally do this synchronously:

// Assume files is already populated with an array of file objects
for(file in files) {
   var currentFile = files[file];
   currentFile.md5 = md5(file.path);
}

The problem here is that the MD5 function is asynchronous and actually has a callback function that is runs once the MD5 hash has been generated for the file. Thus, all of my currentFile.md5 variables are just going to be set to undefined.

Once I have gotten all of the MD5 hashes for all of the files I'll need to move onto another function to deal with that information.

How gnarly is the code going to get in order for me to do this asynchronously? What's the cleanest way to accomplish what I want to do? Are there common different approaches that I should be aware of?

Upvotes: 1

Views: 89

Answers (4)

Salman
Salman

Reputation: 9447

To call an async function multiple times, you should make a function and call it in recursion like this.

I have assumed your md5 function has a callback with two params err and result.

var keys = Object.keys(files); // taking all keys in an array.

function fn() {
    var currentFile = files[keys.shift()];
    md5(currentFile, function (err, result) {
            // Use result, store somewhere

            // check if more files
        if (keys.length) {
            fn();
        } else {
            // done
        }
    });
}

Upvotes: 2

Draculater
Draculater

Reputation: 2278

To avoid "callback hell" you should introduce the world of promises to your Node toolset. I suggest q https://npmjs.org/package/q

Here is a post on SO that can help and give you an idea of the syntax how to use q.js promises to work with multiple asynchronous operations.

You essentially would run all your async functions with defered promises, the .then() chained method would fire when all promises are resolved and the function passed inside then() can process your MD5'd data.

I hope this helps.

Upvotes: 1

Tan Nguyen
Tan Nguyen

Reputation: 3354

To answer your questions (theoretically), in Javascript world, there are (at the moment) 2 different ways to deal with asynchronous code

  • Using callbacks. This is the most basic way that people start using Javascript know. However , there are plenty of libraries to help people deal with callback in a less painful way such as async, step. In your particular problem. Assuming that md5 is somehow weirdly asynchronous, you can use https://github.com/caolan/async#parallel to achieve it

  • Another way is to use promise, there are also plenty of promise-compliant libraries such as q, when. Basically, with a promise you have a nicer way to organize your code flow (IMO). With the problem above you can use when.all to gather the result of md5. However, you need to turn md5 into a promise-compliant function

Upvotes: 1

user949300
user949300

Reputation: 15729

One great approach is to use async. (Search on npm)

If you want to roll your own

  1. Count the files, put that in a var
  2. Everytime fs opens a file and calls your intermediate callback, compute and store the MD5
  3. Also, decrement that counter.
  4. When counter === 0, call a "final" callback, passing back all the MD5s.

Upvotes: 1

Related Questions