CODEWITHSUNDEEP
javascriptasynchronousasync.js
aolivera
aolivera

Reputation: 560

Sequential async operations for each in Node JS

I am working on a database migration. This requires querying one DB, getting an array of records and perform a set of async operations to insert the data in the new DB. In order to keep data consistency, I want to insert the records one at the time so I want each operation to run sequentially. The only way I have found to do these is using recursion.

Is there a cleaner way of doing this same thing? I know there is a library called async https://caolan.github.io/async/v3/ which I never tried before.

The recursive method I have written looks like this:

const insertItem = async (data) => {
    let item = data[0];

    if (!item) {
        //I am done return
        return;
    }

    try {
        //Do multiple await calls to insert record into new database
    } catch (e) {
        //Recover from error (DB rollbacks, etc)
    } finally {
        //Remove inserted or failed item from collection.
        data.shift();
        await insertItem(data);
    }
};

//Query original database
getInfo().then((data) => insertItem(data));

Upvotes: 0

Views: 543

Answers (2)

traktor
traktor

Reputation: 19386

The posted code achieves iterating through the data collection (an array) retrieved from the first data base by using data.shift to mutate the argument array before calling the single function handling everything recursively.

To clean this up, remove the shift and recursive calls by splitting the data processing function into two:

  • one function to step through data retrieved from the first data base,
  • a second function to insert records in the second data base as required.

This removes the need for the .finally clause and leaves the structure of the code looking more like

async function insertData(data) {
    for( let index = 0 ; index < data.length; ++index) {
        await insertItem( data[index]);
    }
}

async function insertItem( data) {
    try {
        //Do multiple await calls to insert record into new database
    } catch (e) {
        //Recover from error (DB rollbacks, etc)
        // throwing an error here aborts the caller, insertData()
    }
}

getInfo().then( insertData).catch( /*... handle fatal error ...*/);

Depending on preferred style, insertItem could be declared as nested function within insertData to keep it looking neat, and insertData could be written as an anonymous function argument of the then call after getInfo().

It is possible, of course, to perform asynchronous operations sequentially be other means, but using await inside an async function is perhaps the simplest coding method.

Upvotes: 0

xMayank
xMayank

Reputation: 1995

You can use sync loop for...of it will wait for HTTP response.

const dataArr = ['data1', 'data2', 'data3'];

async function processItems(arr){
  for(const el of arr) {
    const response = await insertData(el);    
    // add some code here to process the response.
  }
};

processItems(dataArr);

Upvotes: 1

Related Questions