Nester deferred ajax calls in loops

Question

I've got a complicated (at least for me) set up of nested loops, ajax calls, and deferreds. The code is calling an API, parsing out relevant data, then using it to make further calls to other APIs.

It works almost as intended. I used the answer to this question (Using $.Deferred() with nested ajax calls in a loop) to build it. Here's my code:

function a() {
var def = $.Deferred();
var req = [];

for (var i = 0 /*...*/) {
    for (var j = 0 /*...*/) {
        (function(i, j) {
            req.push($.ajax({
                //params
            }).done(function(resp) {
                var def2 = $.Deferred();
                var req2 = [];

                for (var k = 0 /*...*/) {
                    for (var l = 0 /*...*/) {
                        req2.push(b(l));
                    }
                }

                $.when.apply($, req2).done(function() {
                    console.log("Got all data pieces");
                    def2.resolve();
                })
            }));
        })(i, j);
    }
}

$.when.apply($, req).done(function() {
    console.log("Got all data");
    def.resolve();
});

return def.promise();

}

function b(j) {
var def = $.Deferred();

$.when.apply(
    $.ajax({
        //params
    })
).then(function() {
    console.log("Got data piece #" + l);
    def.resolve();
});

return def.promise();

}

function main() {
//...

$.when.apply($, a()).then(function() {
    console.log("All done");
    displayPage();
})

//...

}

Here's what I'm expecting to see when the calls complete

(In no specific order)
Got data piece #1
Got data piece #0
Got data piece #2
Got all data pieces
Got data piece #2
Got data piece #1
Got data piece #0
Got all data pieces
Got data piece #0
Got data piece #1
Got data piece #2
Got all data pieces
Got all data            <-- These two must be last, and in this order
All done

Here's what I'm seeing

All done
Got data piece #0
Got data piece #1
Got data piece #2
Got all data pieces
Got data piece #0
Got data piece #1
Got data piece #2
Got all data pieces
Got data piece #0
Got data piece #1
Got data piece #2
Got all data pieces

I stepped through it in the debugger, and the 'Got all data' line in function a() gets printed in the correct sequence after everything else completes, after which def.resolve() should get called and resolve the returned promise.

However, in main(), a() is seen as resolved right away and the code jumps right into printing 'All done' and displaying the page. Any ideas as to why it doesn't wait as it's supposed to?

Answer 1

You have illustrated a set of code and said it isn't doing what you expected, but you haven't really described the overall problem. So, I don't actually know exactly what code to recommend. We do a lot better here with real problems rather than pseudo code problems. So, instead, what I can do is to outline a bunch of things that are wrong with your code:

Expecting serial order of parallel async operations

Based on what you say you are expecting, the basic logic for how you control your async operations seems to be missing. When you use $.when() on a series of promises that have already been started, you are running a whole bunch of async operations in parallel. Their completion order is completely unpredictable.

Yes, you seem to expect to be able to run a whole bunch of b(i) in parallel and have them all complete in order. That seems to be the case because you say you are expecting this type of output:

Got data piece #0 Got data piece #1 Got data piece #2

where each of those statements is generated by the completion of some b(i) operation.

That simply will not happen (or it would be blind luck if it did in the real world because there is no code that guarantees the order). Now, you can run them in parallel and use $.when() to track them and $.when() will let you know when they are all done and will collect all the results in order. But when each individual async operation in that group finishes is up to chance.

So, if you really wanted each of your b(i) operations to run and complete in order, then you would have to purposely sequence them (run one, wait for it to complete, then run the next, etc...). In general, if one operation does not depend upon the other, it is better to run them in parallel and let $.when() track them all and order the results for you (because you usually get your end result faster by running them all in parallel rather than sequencing them).

Creation of unnecessary deferreds in lots of places - promse anti-pattern

In this code, there is no need to create a deferred at all. $.ajax() already returns a promise. You can just use that promise. So, instead of this:

function b(j) {
    var def = $.Deferred();

    $.when.apply(
        $.ajax({
            //params
        })
    ).then(function() {
        console.log("Got data piece #" + l);
        def.resolve();
    });
    return def.promise();
}

You can do this:

function b(j) {
    return $.ajax({
        //params
    }).then(function(data) {
        console.log("Got data piece #" + l);
        return data;
    });
}

Note, that you just directly return the promise that is already produced by $.ajax() and no deferred needs to be created at all. This is also a lot more bulletproof for error handling. One of the reason your method is called an anti-pattern is you don't handle errors at all (a common mistake when using this anti-pattern). But, the improved code, propagates errors right back to the caller just like they should be. In your version, if the $.ajax() call rejects its promise (due to an error), your deferred is NEVER resolved and the caller never sees the error either. Now, you could write extra code to handle the error, but there is no reason to. Just return the promise you already have. When coding with async operations that return promises, you should pretty much never need to create your own deferred.

$.when() is only needed when you have more than one promise

In your b() function, there is no need to use $.when() in this piece of code:

$.when(
    $.ajax({
        //params
    })).then(...);

When you have a single promise, you just use .then() directly on it.

    $.ajax({
        //params
    }).then(...);

Only use $.when() when you have more than one promise and you want to know when all of them are done. If you only have one promise, just use its own .then() handler.

More anti-pattern - just return promises from .then() handlers

In your inner loop, you have this:

            $.when.apply($, req2).done(function() {
                console.log("Got all data pieces");
                def2.resolve();
            })

There are several things wrong here. It's not clear what you're trying to do because def2 is a deferred that nothing else uses. So, it appears you're trying to tell someone when this req2 group of promises is done, but nobody is using it. In addition it's another version of the anti-pattern. $.when() already returns a promise. You don't need to create a deferred to resolve when $.when() completes. You can just use the promise that $.when() already returns.

Though I don't fully know your intent here, it appears that what you should probably do is to get rid of the def2 deferred entirely and do just this:

            return $.when.apply($, req2).done(function() {
                console.log("Got all data pieces");
            });

Returning this promise from the .then() handler that it is within will chain this sequence of actions to the parent promise and make the parent promise wait for this new promise to be resolved (which is tied to when all the req2 promises are done) before the parent promise will resolve. This is how you make parent promises dependent upon other promise within a .then() handler. You return a promise from the .then() handler.

And, the exact same issue is true for your outer $.when.apply($, req) also. You don't need a deferred there at all. Just use the promise that $.when() already returns.

Putting it together

Here's a cleaned up version of your code that gets rid of the anti-patterns in multiple places. This does not change the sequencing of the b(i) calls among themselves. If you care about that, it is a bigger change and we need to see more of the real/actual problem to know what best to recommend.

function a() {
    var req = [];

    for (var i = 0 /*...*/) {
        for (var j = 0 /*...*/) {
            (function(i, j) {
                req.push($.ajax({
                    //params
                }).then(function(resp) {
                    var req2 = [];

                    for (var k = 0 /*...*/) {
                        for (var l = 0 /*...*/) {
                            req2.push(b(l));
                        }
                    }

                    return $.when.apply($, req2).done(function() {
                        console.log("Got all data pieces");
                    });
                }));
            })(i, j);
        }
    }

    return $.when.apply($, req).done(function() {
        console.log("Got all data");
    });
}    

function b(j) {
    return $.ajax({
            //params
    }).then(function(data) {
        console.log("Got data piece #" + l);
        return data;
    });
}

function main() {
    //...

    a().then(function() {
        console.log("All done");
        displayPage();
    });

    //...
}

P.S. If you want to process the b(i) results from within the same group in order, then don't use a .then() handler on the individual promise because those will execute in arbitrary order. Instead, use the results that come with $.when().then(result1, result2, ...) and process them all there. Though the individual promises complete in an arbitrary order, $.when() will collect the results into the original order so if you process the results in the $.when() handler, then you can process them all in order.