Node.JS MongoDB find: Sequential faster than parallel, why?

Question

I have encountered a strange behaviour that contradicts what I thought I'd know about mongodb:

I have 4 empty collections. I do a find on each collection and measure, how long the queries will take until being fulfilled.

For the "parallel"-test, I start all queries using await Promise.all(queries). For the "sequential"-test, I start all queries using "await queries[0..3]".

I would expect that, due to the collections being empty, the measured times are in general quite low, with the parallel request executing a bit faster than the sequential one. The queries go towards localhost - meaning that transporting the data from the client to the mongodb server and back should not add much overhead.

However; the results:

parallel: 2008 ms

sequential: 2 ms

Maybe my code has issues?

When I execute the queries multiple times in a loop, I receive the following results:

Parallel 1: 2008.642 ms

Sequential 1: 2.344 ms

Parallel 2: 1004.544ms

Sequential 2: 5.273ms

Parallel 3: 2.152ms

Sequential 3: 3.605ms

Parallel 4: 2.189ms

Sequential 4: 3.885ms

Whenever I restart the program, I receive similar results (with and without dropping/recreating the collections).

My question: Why does the parallel queries need more time than the sequential ones? And why do the parallel requests speed up with each request done, but reset the request duration when I restart the Node.JS software?

Due to the behaviour, I expect the reason to lay somewhere in my Node.JS code or in the MongoDB Node.JS driver.

My code:

import {
    MongoClient,
    ObjectID,
} from 'mongodb';

const run = async () => {
    const client = await MongoClient.connect('mongodb://localhost:27017/mydatabase', {
            useNewUrlParser: true
        }),
        db = client.db('mydatabase'),
        userId = new ObjectID();

    // create collections
    await db.dropCollection('collection1');
    await db.dropCollection('collection2');
    await db.dropCollection('collection3');
    await db.dropCollection('collection4');
    await db.createCollection('collection1');
    await db.createCollection('collection2');
    await db.createCollection('collection3');
    await db.createCollection('collection4');

    // measure read fullfill times
    for (let i = 1; i < 5; ++i) {
        console.time(`Parallel ${i}`);
        await Promise.all([
                db.collection('collection1')
                    .find()
                    .toArray(),

                db.collection('collection2')
                    .find()
                    .toArray(),

                db.collection('collection3')
                    .find()
                    .toArray(),

                db.collection('collection4')
                    .find()
                    .toArray(),
        ]);
        console.timeEnd(`Parallel ${i}`);

        console.time(`Sequential ${i}`);
        await db.collection('collection1')
            .find()
            .toArray();

        await db.collection('collection2')
            .find()
            .toArray();

        await db.collection('collection3')
            .find()
            .toArray();

        await db.collection('collection4')
            .find()
            .toArray();
        console.timeEnd(`Sequential ${i}`);
    }
};

run();

The compiled code can be found here: https://pastebin.com/ETmPPbzd

Best regards

Answer 1

Due to the behaviour, I expect the reason to lay somewhere in my Node.JS code or in the MongoDB Node.JS driver.

Yes, you're right! It's within the mongo-client pool. When you first create the pool, it has a single connection. When you ask 4 requests at once, it tries opening up more connections and generally seems to get into a bad state. http://mongodb.github.io/node-mongodb-native/core/driver/reference/pool/

If you set your poolSize to 1 for your connection, you should see similar performance between parallel & serial. There's also a minSize option that will initialize the connection pool with the appropriate number of connections & ensure that the pool-size never dips below the minimum size.