CODEWITHSUNDEEP
javascriptnode.jsasynchronouscallback
Lazarus Rising
Lazarus Rising

Reputation: 2685

Make callback wait till function returns result in node js

There is an array of files that will have to be processed. Files may be of different types, so there are different functions doing the processing. I expect to go through all files, collect logs while at it, and in the end return a variable where all the logs are stored.

The code:

var file_list = req.body.filelist.split(','); //array of filenames
var logs = []; //logs about the file processing will be stored here

//iteration through each file
async.forEach(file_list , function(file_from_list , callback){
    if(file_is_type1(file_from_list)){
        process_filetype_1(file_from_list);
        callback(null);
    }
    else if(file_is_type1(file_from_list)){
        process_filetype_2(file_from_list);
        callback(null);
    }
    else{
        logs.push("Unknown file type");
    }
},function(error, result){
     return res.status(200).send({message: import_log});
});

There are two different functions with differences in logic that process the files. It is a complicated code, but runs smoothly and each action is performed at the right time. Here is an abstraction of one of the functions:

function process_filetype_1(file_from_list){
    async.auto({
        dosth_1: function(callback) {
            //logic implementation
        },
        dosth_2: ['dosth_1', function(callback) {
            //logic implementation
        }]
    }, function(error, results) {
        return results;
    });
}

What happens: all files are processed, but callbacks inside the iteration don't wait for the function to finish execution. Therefore the response is send before files have finished processing

Why it happens: calling process_filetype_1(file_from_list) and callback(null) is asynchronous. callback(null) finishes first, all files are iterated, the execution flow goes to sending the response. In the meanwhile the functions are still executing.

How to fix it: Haven't figured it out. Ideally, callback would wait for the functions to return their respective values (and finish executing) before returning null.

What cannot be done: using timers to make callback wait. The number of files being processed could be from 1 to hundreds, and there is no limit in size. The amount of time it may take to process each file is unknown.

What I have tried: returning callbacks when the function finishes executing. Assigning the function's return value to a variable (ex. test) and doing callback(test). Neither one worked.

Upvotes: 1

Views: 2106

Answers (2)

markS
markS

Reputation: 580

you could try something like:

if(file_is_type1(file_from_list)){
   process_filetype_1(file_from_list, callback);
}

and then process_filetype_1 would look something like:

function process_filetype_1(file_from_list, callback){
    async.auto({
        dosth_1: function(cb) {
            //logic implementation
        },
        dosth_2: ['dosth_1', function(cb) {
            //logic implementation
        }]
    }, function(error, results) {
        if(error) { callback (error); 
        } else {
        callback(results); 
        }
    });
}

Upvotes: 2

tadman
tadman

Reputation: 211590

If you want to use callback-style code then any function which makes calls that take callbacks must itself take a callback function:

function process_filetype_1(file_from_list, cb) {
    async.auto({
        dosth_1: function(callback) {
            //logic implementation
        },
        dosth_2: ['dosth_1', function(callback) {
            //logic implementation
        }]
    }, function(error, results) {
        cb(error, results);
    });
}

Since your function doesn't seem to do anything you can just chain through:

function process_filetype_1(file_from_list, cb) {
    async.auto({
        dosth_1: function(callback) {
            //logic implementation
        },
        dosth_2: ['dosth_1', function(callback) {
            //logic implementation
        }]
    }, cb);
}

An alternative is to use the Promise system possibly in conjunction with async/await to help detangle this code. I've found they're considerably easier to work with, especially with complicated flow control situations.

Upvotes: 2

Related Questions