Update pattern for progress bar on a long ajax query

Question

Here's what I have:

1. a Node.js function that performs a long (2-10 seconds) calculation (db + external API calls). I use async.auto() to organize my calls' dependencies
2. a nice visual progress bar on the HTML view page
3. a getJSON function calling the query (see code below)

Right now, I'm just updating the progress bar on a timer. But I'd like it to be more accurate. I already broke the server side function to 5 milestones (i.e. after every external call finishes successfully), and I'd like to send that progress info to the client. However, once you res.send(progress) you complete the response and nothing more could be sent on it.

I obviously need a different pattern here, and I had some ideas:

1. 2 ajax calls, one polls for progress and the second gets the results
2. one call that gets called 5 times, until the server side function is gone
3. Something completely different (maybe long polling, websockets, ?)

The question:

I'd like to know what's the recommended pattern, and how to implement it on both the server and client side. Sample code, or links to samples, are highly appreciated!

And, as always, thanks for your time!

Included here is the code I have so far (with obvious functionality omissions). It's not strictly necessary to answer this question, but I provide it here to give some context.

Client

var searchFunction = function() {
        var user = $("#searchTerm").val();
        $.getJSON("/search/" + user, function(data) {
            console.log(data);
            ko.applyBindings(new MyViewModel(data));
        });
        $("#content").hide();
        progressBar(5);
        $("#progress_bar").show();
        setTimeout(function() {
            progressBar(25);
        }, 1000);
        setTimeout(function() {
            progressBar(50);
        }, 2000);
        setTimeout(function() {
            progressBar(75);
        }, 3000);
        setTimeout(function() {
            $("#progress_bar").hide();
            $("#content").show();   
        }, 4000);
    };

Server

var searchFunction = function(req, res) {   
    async.auto({
        f1: function(callback) {
            //things happen
            callback(err, res);
        },
        f2: function(callback) {
            //things happen
            callback(err, res);
        },
        f3: ['f2', function(callback, result) {
            //many things happen
            callback(err, res);
        }, 
        function(err) {
            //decisions are made
            callback(null, watchers);
        }]  
    }, function(err, result) {  
            //more exciting stuff happens       
            res.send(finalResults);
        }
    );
};

Answer 1

You could use an event stream to push a percent-complete value back up to the client. But it only works in modern browsers. IE9+ I believe. Here's an example event stream: https://github.com/chovy/nodejs-stream

html

<div id="progress"><span class="bar"></span></div>

css

#progress { width: 200px; background: #eee; border: 1px solid #ccc; height: 20px; }
#progress .bar { display: inline-block; background: #ddd; width: 0; }

client side (jQuery) - listen to /progress event stream

var source = new EventSource('progress');
source.addEventListener('progress', function(e) {
  var percentage = JSON.parse(e.data),
  //update progress bar in client
  $("#progress .bar").css({ width: percentage });
}, false);

server side - push an event with new progress percent

res.writeHead(200, {'Content-Type': 'text/event-stream'});
res.write("event: progress\n");
//push the percentage to the client (ie: "10%")
res.write("data: "+"10%".toString()+"\n\n");

It would be up to you to define your progress bar in the UI --- I would use a fixed width div with a child that has css rule w/ percentage-based width. That gets updated.

Answer 2

My first thought would be to use socket.io or something similar, with the server emitting an event at each milestone. The client-side version of searchFunction() would simply make the AJAX call and then return, with the socket handler taking care of updating the progress bar.