CODEWITHSUNDEEP
node.jsmongodb
Houssam-Hamza BERRAHAOUI
Houssam-Hamza BERRAHAOUI

Reputation: 11

Insert data after each iteration in a FOR Loop

I'm trying to insert data to a MongoDB database via node.js

The function addToDatabase is invoked from a for loop. Regardless of multiple iterations, the script don't insert data into database until its done with all iterations (at the end of for loop).

I just want the script to complete the database insert operation after each iteration.

Thanks in advance.

Here is my code:

const mongoClient = require('mongodb').MongoClient;

function addToDatabase(data){
    MongoClient.connect(config.database.url, {useNewUrlParser: true}, function(err, db) {
        if (err) throw err;
        var dbo = db.db(config.database.dbName);
        dbo.collection(config.database.collectionAd).insertOne(data, function(err, res) {
            if (err) throw err;
            console.log('Ad insered : ' + data._id);
            db.close();
        });
    });

}

function getAdDetails(ads){
    for (var itemKey in ads.items) {
        //Statements ...
        addToDatabase(obj);
    }
}

getAdDetails(obj)

Upvotes: 0

Views: 256

Answers (1)

Khaled Osman
Khaled Osman

Reputation: 1467

There're a few things not optimal in this code.

  1. You're connecting and disconnecting from the database on each iteration in the loop which is alot of load on the database and not performant. You should cache/reuse the connection and connect once at the beginning and disconnect at the end once all the work is done.
  2. instead of looping and using insertOne, you can use insertMany to minimize load on the database and to be more performant https://docs.mongodb.com/manual/reference/method/db.collection.insertMany/
  3. Instead of using callbacks, MongoDB client already returns promises so you can use that to wait when all the operations are done before closing the connection
  4. (Probably the main issue) You're closing the connection inside the callback of one of the queries in a for loop, since its asynchronous, you might be closing the connection mid operation before all operations are done and connecting synchronously as in a random pattern like "connect connect connect, disconnect connect connect, disconnect, etc.." not "connect disconnect connect disconnect" as you would expect. to fix this you can make sure addToDatabase returns a promise that resolves once the insert is successful and use Promise.all to wait until all inserts are done

try something like this instead

const mongoClient = require('mongodb').MongoClient

(async function () {
  // connect
  try {
    await mongoClient.connect(config.database.url, { useNewUrlParser: true })
    const db = mongoClient.db(config.database.dbName)

    // do stuff
    const results = await db.collection(config.database.collectionAd).insertMany(ads.items)
    console.log('Ads insered : ', results)
  } catch (err) {
    console.log(err)
  }
  // close connection at the end
  mongoClient.close()
})()

Upvotes: 1

Related Questions