CODEWITHSUNDEEP
node.jssequelize.js
troyz
troyz

Reputation: 1385

Sequelize commit after every insert

I'm importing data from a CSV file into a Nodejs system using Sequelize, but I'm having issues trying to check if a previous row has been inserted before.

This is the code I'm using

exports.insertEmployee = function (employee) {
    return new Promise((fulfill, reject) => {
        models.Employee.create(employee)
        .then(employee => fulfill(employee.dataValues))
        .catch(reject);
    });
};

exports.importEmployee = function (employee) {
    return new Promise((fulfill, reject) => {
        models.Employee.findOne({where: {customID: employee.customID}, raw: true}).then(emp => {
            if(emp) {
                fulfill(emp);
            }
            else {
                exports.insertEmployee(employee).then(fulfill);
            }
         });
    });
};

The function importEmployee will be called as many times as lines there are in the csv like this

lines.forEach(line => {
    let emp = line.employee;
    importEmployee(emp).then(employee => {console.dir(employee)});
});

The problem I'm having is that when I check if an user with the same customID has been inserted already, the database hasn't commited the insert yet so it will always return false and I'll have duplicated employees with the same customID

How could I fix that? I tried using transactions but maybe I'm doing it wrong because it won't help.

Thanks!

Upvotes: 0

Views: 1374

Answers (1)

piotrbienias
piotrbienias

Reputation: 7401

It did not work because inside your forEach you wanted to perform series of asynchronous operations. In this way all the records will be inserted because those operations performing check and insert are running concurrently. In order to avoid this situation you can use the mapSeries() Promise function from Bluebird module. It is built in into the sequelize so there is no need to install additional package.

Moreover, you can perform the check and insert via single operation called findOrCreate(), which returns an instance and boolean defining if this instance was created or simply returned as an existing one.

When combining both those functions, your operation becomes very simple and straightforward

// here is the function performing findOrCreate
exports.createEmployee = function(employee) {
    return models.Employee.findOrCreate({
        where: { customID: employee.customID },
        defaults: employee
    });
};

What is more, you do not have to create nested Promises, just like you did in insertEmployee and importEmployee. Most of sequelize functions already return promises, so you can simply return the function call itself.

Now you can iterate over lines and call the createEmployee method.

models.sequelize.Promise.mapSeries(lines, (employee) => {
    return createEmployee(employee);
}).then(result => {
    console.log(result);
    // example output: [ [ Instance, true ], [ Instance, true ], [ Instance, false ] ] 
});

The boolean true means that the instance was created, whereas false means that instance was returned from table because such record already exists.

Upvotes: 2

Related Questions