Javascript async function composition

Question

I have several async functions with varying numbers of parameters, in each the last param is a callback. I wish to call these in order. For instance.

function getData(url, callback){
}
function parseData(data, callback){
}

By using this:

Function.prototype.then = function(f){ 
  var ff = this; 
  return function(){ ff.apply(null, [].slice.call(arguments).concat(f)) } 
}

it is possible to call these functions like this, and have the output print to console.log.

getData.then(parseData.then(console.log.bind(console)))('/mydata.json');

I've been trying to use this syntax instead, and cannot get the Then function correct. Any ideas?

getData.then(parseData).then(console.log.bind(console))('/mydata.json');

Answer 1

This looks like an excellent use for the Promise object. Promises improve reusability of callback functions by providing a common interface to asynchronous computation. Instead of having each function accept a callback parameter, Promises allow you to encapsulate the asynchronous part of your function in a Promise object. Then you can use the Promise methods (Promise.all, Promise.prototype.then) to chain your asynchronous operations together. Here's how your example translates:

// Instead of accepting both a url and a callback, you accept just a url. Rather than
// thinking about a Promise as a function that returns data, you can think of it as
// data that hasn't loaded or doesn't exist yet (i.e., promised data).
function getData(url) {
    return new Promise(function (resolve, reject) {
        // Use resolve as the callback parameter.
    });
}
function parseData(data) {
    // Does parseData really need to be asynchronous? If not leave out the
    // Promise and write this function synchronously.
    return new Promise(function (resolve, reject) {
    });
}
getData("someurl").then(parseData).then(function (data) {
    console.log(data);
});

// or with a synchronous parseData
getData("someurl").then(function (data) {
    console.log(parseData(data));
});

Also, I should note that Promises currently don't have excellent browser support. Luckily you're covered since there are plenty of polyfills such as this one that provide much of the same functionality as native Promises.

Edit:

Alternatively, instead of changing the Function.prototype, how about implementing a chain method that takes as input a list of asynchronous functions and a seed value and pipes that seed value through each async function:

function chainAsync(seed, functions, callback) {
    if (functions.length === 0) callback(seed);
    functions[0](seed, function (value) {
        chainAsync(value, functions.slice(1), callback);
    });
}
chainAsync("someurl", [getData, parseData], function (data) {
    console.log(data);
});

Edit Again:

The solutions presented above are far from robust, if you want a more extensive solution check out something like https://github.com/caolan/async.

Answer 2

Robert Rossmann is right. But I'm willing to answer purely for academic purposes.

Let's simplify your code to:

Function.prototype.then = function (callback){ 
  var inner = this;
  return function (arg) { return inner(arg, callback); }
}

and:

function getData(url, callback) {
    ...
}

Let's analyze the types of each function:

• getData is (string, function(argument, ...)) → null.
• function(argument, function).then is (function(argument, ...)) → function(argument).

That's the core of the problem. When you do:

getData.then(function (argument) {}) it actually returns a function with the type function(argument). That's why .then can't be called onto it, because .then expects to be called onto a function(argument, function) type.

What you want to do, is wrap the callback function. (In the case of getData.then(parseData).then(f), you want to wrap parseData with f, not the result of getData.then(parseData).

Here's my solution:

Function.prototype.setCallback = function (c) { this.callback = c; }
Function.prototype.getCallback = function () { return this.callback; }

Function.prototype.then = function (f) {
  var ff = this;
  var outer = function () {
     var callback = outer.getCallback();
     return ff.apply(null, [].slice.call(arguments).concat(callback));
  };

  if (this.getCallback() === undefined) {
    outer.setCallback(f);
  } else {
    outer.setCallback(ff.getCallback().then(f));
  }

  return outer;
}

Answer 3

The problem is that then returns a wrapper for the current function and successive chained calls will wrap it again, instead of wrapping the previous callback. One way to achieve that is to use closures and overwrite then on each call:

Function.prototype.then = function(f){ 
  var ff = this;

  function wrapCallback(previousCallback, callback) {
    var wrapper = function(){ 
      previousCallback.apply(null, [].slice.call(arguments).concat(callback)); 
    };

    ff.then = wrapper.then = function(f) {
      callback = wrapCallback(callback, f); //a new chained call, so wrap the callback
      return ff;    
    }

    return wrapper;
  }
  
  return ff = wrapCallback(this, f); //"replace" the original function with the wrapper and return that
}

/*
 * Example
 */ 
function getData(json, callback){
    setTimeout( function() { callback(json) }, 100);
}

function parseData(data, callback){
   callback(data, 'Hello');
}

function doSomething(data, text, callback) {
  callback(text);  
}

function printData(data) {
  console.log(data); //should print 'Hello'
}

getData
    .then(parseData)
    .then(doSomething)
    .then(printData)('/mydata.json');

Answer 4

I had some thoughts about that problem and created the following code which kinda meets your requirements. Still - I know that this concept is far away from perfect. The reasons are commented in the code and below.

Function.prototype._thenify = {
    queue:[],
    then:function(nextOne){
        // Push the item to the queue
        this._thenify.queue.push(nextOne);
        return this;
    },
    handOver:function(){
        // hand over the data to the next function, calling it in the same context (so we dont loose the queue)
        this._thenify.queue.shift().apply(this, arguments);
        return this;
    }
}

Function.prototype.then = function(){ return this._thenify.then.apply(this, arguments) };
Function.prototype.handOver = function(){ return this._thenify.handOver.apply(this, arguments) };

function getData(json){
    // simulate asyncronous call
    setTimeout(function(){ getData.handOver(json, 'params from getData'); }, 10);
    // we cant call this.handOver() because a new context is created for every function-call
    // That means you have to do it like this or bind the context of from getData to the function itself
    // which means every time the function is called you have the same context
}

function parseData(){
    // simulate asyncronous call
    setTimeout(function(){ parseData.handOver('params from parseData'); }, 10);
    // Here we can use this.handOver cause parseData is called in the context of getData
    // for clarity-reasons I let it like that
}

getData
    .then(function(){ console.log(arguments); this.handOver(); }) // see how we can use this here
    .then(parseData)
    .then(console.log)('/mydata.json');                           // Here we actually starting the chain with the call of the function
    

// To call the chain in the getData-context (so you can always do this.handOver()) do it like that:
// getData
//     .then(function(){ console.log(arguments); this.handOver(); })
//     .then(parseData)
//     .then(console.log).bind(getData)('/mydata.json');

Problems and Facts:

• the complete chain is executed in the context of the first function
• you have to use the function itself to call handOver at least with the first Element of the chain
• if you create a new chain using the function you already used, it will conflict when it runs to the same time
• it is possible to use a function twice in the chain (e.g. getData)
• because of the shared conext you can set a property in one function and read it in one of the following functions

At least for the first Problem you could solve it with not calling the next function in the chain in the same context and instead give the queue as parameter to the next function. I will try this approach later. This maybe would solve the conflicts mentioned at point 3, too.

For the other problem you could use the sample Code in the comments

PS: When you run the snipped make sure your console is open to see the output

PPS: Every comment on this approach is welcome!

Answer 5

Implementing a function or library that allows you to chain methods like above is a non-trivial task and requires substantial effort. The main problem with the example above is the constant context changing - it is very difficult to manage the state of the call chain without memory leaks (i.e. saving a reference to all chained functions into a module-level variable -> GC will never free the functions from memory).

If you are interested in this kind of programming strategy I highly encourage you to use an existing, established and well-tested library, like Promise or q. I personally recommend the former as it attempts to behave as close as possible to ECMAScript 6's Promise specification.

For educational purposes, I recommend you take a look at how the Promise library works internally - I am quite sure you will learn a lot by inspecting its source code and playing around with it.