CODEWITHSUNDEEP
javascriptnode.jsmeteor
Nyxynyx
Nyxynyx

Reputation: 63687

Meteor._wrapAsync causes Meteor to be unresponsive

I'm using Meteor._wrapAsync to force only one call to the function writeMeLater to be executing at any one time. If 10 calls to writeMeLater are made within 1 second, the other 9 calls should be queued up in order.

To check that writeMeLater is running synchronously, the timestamp field in the Logs Collection should be spaced 1 second apart.

Problem: With the following code, only the first call to writeMeLater is executed, the other 9 does not appear to run. Why is this so?

Server Code:

writeMeLater = function(data) {
    console.log('writeMeLater: ', data)

    // simulate taking 1 second to complete
    Meteor.setTimeout(function() {
        Logs.insert({data: data, timestamp: new Date().getTime()})
    }, 1 * 1000)
}

writeMeLaterSync = Meteor._wrapAsync(writeMeLater)


// simulate calling the function many times real quick
for(var i=0; i<10; i++) {
    console.log('Loop: ', i)
    writeMeLaterSync(i)
}

Output:

=> Meteor server running on: http://localhost:4000/
I20140119-11:04:17.300(8)? Loop:  0
I20140119-11:04:17.394(8)? writeMeLater:  0

Using an alternate version of writeMeLater, I get the same problem:

writeMeLater = function(data) {
    console.log('writeMeLater: ', data)
    setTimeout(Meteor.bindEnvironment( function() {
        Logs.insert({data: data, timestamp: new Date().getTime()})
    }), 1 * 1000)
}

Upvotes: 3

Views: 427

Answers (1)

saimeunt
saimeunt

Reputation: 22696

TL;DR - your writeMeLater function needs to take a callback parameter.

NodeJS classic asynchronous functions usually have this signature :

function async(params..., callback) {
    try {
        var result = compute(params);
        callback(null,result);
    }
    catch {
        callback("something went wrong", null);
    }
}

They take any number of parameters, the last one being a callback to be run when the computation is ready, called with 2 parameters: error which is null if everything is OK, and the result of course.

Meteor._wrapAsync expects to be given a function with this signature to return a newly pseudo-synchronous function. Meteor "synchronous" functions allows you to write code in a synchronous style, but they are not truly synchronous like NodeJS fs.readFileSync for example, which BLOCKS the event loop until it's done (usually this is bad unless you're writing a command-line app, which is not the case with Meteor).

Note: using NodeJS fs *Sync functions in Meteor is bad because you might be tricked into thinking they are "Meteor synchronous" but they aren't, they will block your entire node process until they're done ! You should be using fs async funcs wrapped with Meteor._wrapAsync.

A simplified clone of Meteor._wrapAsync would look like this:

var wrapAsync=function(asyncFunc) {
    // return a function who appears to run synchronously thanks to fibers/future
    return function() {
        var future = new Future();
        // take the arguments...
        var args = arguments;
        // ...and append our callback at the end
        Array.prototype.push.call(args, function(error, result) {
            if (error) {
                throw error;
            }
            // our callback calls future.return which unblocks future.wait
            future.return(result);
        });
        // call the async func with computed args
        asyncFunc.apply(null, args);
        // wait until future.return is called
        return future.wait();
    };
};

There is a Future.wrap which does exactly this, Meteor._wrapAsync is a bit more complicated because it handles Meteor environment variables by using Meteor.bindEnvironment.

Fibers and Futures are a bit out of scope so I won't dive into them, be sure to check eventedmind.com videos on the subject.

Introducing Fibers - https://www.eventedmind.com/feed/BmG9WmSsdzChk8Pye

Using Futures - https://www.eventedmind.com/feed/kXR6nWTKNctKariSY

Meteor._wrapAsync - https://www.eventedmind.com/feed/Ww3rQrHJo8FLgK7FF

Now that you understand how things need to be done to encapsulate async functions in Meteor, let's fix your code.

If your async function doesn't take a callback as last argument, it won't call it (obviously), and the callback we pass to it in the wrapped function won't trigger either, which means future.return won't be called and this is why your program is blocked in the first place !

You simply have to rewrite writeMeLater to take a callback as final argument :

var writeMeLater = function(data, callback){
    console.log('writeMeLater: ', data);
    // simulate taking 1 second to complete
    Meteor.setTimeout(function() {
        Logs.insert({
            data:data,
            timestamp:new Date().getTime()
        });
        callback(null, "done processing " + data);
    }, 1 * 1000);
};

And you're good to go !

Upvotes: 5

Related Questions