CODEWITHSUNDEEP
node.jspromiseaws-lambda
Amiga500
Amiga500

Reputation: 6131

Handling promises inside the forEach loop

I am trying to run a series of tasks. Each task is dynamic, and could have different rules to follow. This will be executed on AWS-Lambda.

I have an array of JSON. It has a body with task name in it, and it also has attributes.

I need to dynamically load a javascript file with the name inside the body. I need to wait until all is finished inside that task. Or it failed (regardless where). If the fail happens, I will need to write that data inside the current record inside the forEach loop.

I have old issue, where my forEach is finished first without waiting for the task to complete. This is the forEach loop:

const jobLoader = require('./Helpers/jobLoader');

event.Records.forEach(record => {
    const { body: jobName } = record;
    const { messageAttributes } = record;

    const job = jobLoader.loadJob(jobName);

    job.runJob(messageAttributes).then(res => {
        console.log('Show results');
        return; // resume another record from forEach
    }).catch(err => {
         record.failed = true;
         record.failureMessage = err.message;
        console.log('I errored');
    });

    console.log('All Done');
});

The problem is that message All Done is printed, and then the message show results is printed. I get results from the database once it comes for execution.

This is the file that loads a task:

exports.loadJob = (jobName) => {
    const job = require(`../Tasks/${jobName}`);

    return job;
};

This is the file that contains actual task: const mySqlConnector = require('../Storage/mySql');

exports.runJob = async (params) => {
  let payload = {};

    let dataToSend = await getUserName(params.userId.stringValue);

    payload.dataToSend = dataToSend;

    let moreDataToSend = await getEvenMoreData(params.userId.stringValue);

    payload.moreDataToSend = moreDataToSend;

    return await sendData(payload);
};

const getUserName = async (userId) => {
    const query = 'SELECT * FROM user_data';
    return await mySqlConnector.handler(query);
};

const getEvenMoreData = async (userId) => {
    const query = 'SELECT * FROM user_data';
    return await mySqlConnector.handler(query);
};


const sendData = (payload) => {

  //this should be Axios sending data
};

And this is the mySql connector itself: const mysql = require('promise-mysql');

exports.handler = async (query) => {
   return mysql.createConnection({
        host     : '127.0.0.1',
        user     : 'root',
        password : '',
        database: 'crm'
    }).then(conn =>{
        let result = conn.query(query);
        conn.end();
        return result;
    }).then(rows => {
        //console.log("These are rows:" + rows);
        return rows;
    }).catch(error => {
        return error;
    });
};

The task file can have any number of things it needs to complete, which will be different when I start adding tasks. I need that job.runJob completes, or that it catches an error, from whatever location it originated, so I can continue with the forEach.

I have tried using map and what not, but the end result is always the same.

What am I doing wrong?

Upvotes: 1

Views: 1409

Answers (2)

Amiga500
Amiga500

Reputation: 6131

I managed to solve it, and still keep details about the execution:

Something like this:

for (let prop in event.Records){

        const { body: jobName } = event.Records[prop];
        const { messageAttributes } = event.Records[prop];

        const job = jobLoader.loadJob(jobName);

        await job.runJob(messageAttributes).then(res => {
            console.log('Show results', res);
        }).catch(err => {
            event.Records[prop].failed = true;
            event.Records[prop].failed = err.message;
            console.log('I errored');
        });

    }

Upvotes: 1

Julien METRAL
Julien METRAL

Reputation: 1974

You can use Promise.all method :

const promises = event.Records.map(record => {
    const { body: jobName } = record;
    const { messageAttributes } = record;

    const job = jobLoader.loadJob(jobName);

    return job.runJob(messageAttributes).then(res => {
        console.log('Show results', res);
    }).catch(err => {
         record.failed = true;
         record.failureMessage = err.message;
        console.log('I errored');
        throw new Error('Your error !');
    });
});

try {
   const results = await Promise.all(promises);
   console.log('All done');
} catch (e) {
   console.log('Something has an error', e);
}

don't forget to make your function async !

Upvotes: 2

Related Questions