MongoError: pool is draining, new operations prohibited when using MongoMemoryServer in integration test

Question

I'm using MongoMemoryServer to write an integration test. I have two integration test files. When I run the IT tests I see the following. I don't understand why. I'm using jestjs test framework.

I'm seeing the following error when I have two IT test files

MongoError: pool is draining, new operations prohibited

      37 |   for (const key in collections) {
      38 |     const collection = collections[key];
    > 39 |     await collection.deleteMany();
         |                      ^
      40 |   }
      41 | };

Here is my setup

//db-handler.js
const mongoose = require("mongoose");
const { MongoMemoryServer } = require("mongodb-memory-server");
const mongod = new MongoMemoryServer();

module.exports.connect = async () => {
  const uri = await mongod.getConnectionString();

  const mongooseOpts = {
    useNewUrlParser: true,
    autoReconnect: true,
    reconnectTries: Number.MAX_VALUE,
    reconnectInterval: 1000,
  };

  await mongoose.connect(uri, mongooseOpts);
};

module.exports.closeDatabase = async () => {
  await mongoose.connection.dropDatabase();
  await mongoose.connection.close();
  await mongod.stop();
};

module.exports.clearDatabase = async () => {
  const collections = mongoose.connection.collections;

  for (const key in collections) {
    const collection = collections[key];
    await collection.deleteMany();
  }
};

All my IT tests setup looks like this

//example.it.test.js
const supertest = require("supertest");
const dbHandler = require("./db-handler");
const app = require("../../src/app");
const request = supertest(app);

const SomeModel = require("../Some");
beforeAll(async () => await dbHandler.connect());
afterEach(async () => await dbHandler.clearDatabase());
afterAll(async () => await dbHandler.closeDatabase());

describe("Some Test Block", () => {
  it("Test One", async (done) => {
    await SomeModel.create({a: "a", b "b"});

    const response = await request.get("/endPointToTest");
    expect(response.status).toBe(200);

    done();
  });

When I have just a single IT test files, everything works fine. When I introduce a a new IT test file similar setup up as example.it.test.js, then the new test fails. The example error message above.

What am I missing? Does my setup need to change when I have multiple IT test files?

UPDATE ONE: My package.json file looks like

{
  "scripts": {
    "start": "node index.js",
    "dev": "nodemon index.js",
    "test": "jest --runInBand ./tests"
  },
  "devDependencies": {
    "jest": "^25.2.7",
    "mongodb-memory-server": "^6.5.2",
    "nodemon": "^2.0.2",
    "supertest": "^4.0.2"
  },
  "jest": {
    "testEnvironment": "node",
    "coveragePathIgnorePatterns": [
      "/node_modules/"
    ]
  }
}

Answer 1

I think the error message can be somewhat misleading for some cases.

It seems like the issue is related to "pool size being too small" or "jest runs tests in parallel", so the pool eventually ran out of usable pool.

However, if you look at what happens when mongo destroys the pool, you'll get the idea.

/**
 * Destroy pool
 * @method
 */
Pool.prototype.destroy = function(force, callback) {
  ...

  // Set state to draining
  stateTransition(this, DRAINING);

  ...

If you try to close the connection, the pool changes its internal state into 'draining' which causes the problem.

So what the error message is telling us is that there's still unfinished writing operation(s) going on after pool.destroy.

For my case, that was an active event listener callback runs every x seconds which does write to mongo.

I changed my test code to call event callback function directly instead of being called every x seconds.

That resolved my issue.

Answer 2

For every It test case, it connects to the mongoose, and if any error occurred, mongoose does not disconnect.

Solution

1. You can put the it test case in the try catch block, but if possible use 2 point
2. Try to handle the error in the controllers and service, and so on files if possible.
3. And whenever you write test case, there should be no error comes from the development code.
4. Don't write nested describe in a single describe, max 2 nested describe we should use.
5. One temporary solution, I skip the all describe expect outmost where mongoose was connected.
6. One more temporary solution, disconnect the mongod and then see the status, then reconnect.

Answer 3

I did the same way and faced into the same error. This was resolved by removing by schema name in test only like I did below in some.test.js

** Rest all setup remains same **

//example.it.test.js
const dbHandler = require("./db-handler");
const SomeModel = require("../Some");
const SomeOtherModel = require("../SomeOther");
beforeAll(async () => await dbHandler.connect());
afterEach(async () => {
  await SomeModel.remove({});
  await SomeOtherModel.remove({});
});
afterAll(async () => await dbHandler.closeDatabase());

describe("Some Test Block", () => {
  it("Test One", async (done) => {
    await SomeModel.create({a: "a", SomeOther: 'SomeOther Data' });
    const data = await SomeModel.getAll();
    expect(data.length).toBe(1);
    done();
  });
});

Answer 4

By default Jest runs tests in parallel with a “a worker pool of child processes that run tests” (Jest CLI docs). As per the Jest documentation.

Each test file is making a new connection to the mogodb server which is causing the error.

You could run the Jest test sequentially to avoid this issue.

The --runInBand or -i option of the Jest CLI allows you to run tests sequentially (in non-parallel mode).

This error (pool is draining, new operations prohibited) might occur if you are using the official mongodb library for node js or mongoose. Please specifying pool size while establishing mongodb connection.

module.exports.connect = async () => {
  const uri = await mongod.getConnectionString();
  const mongooseOpts = {
    useNewUrlParser: true,
    autoReconnect: true,
    reconnectTries: Number.MAX_VALUE,
    reconnectInterval: 1000,
    poolSize: 10,
  };
  await mongoose.connect(uri, mongooseOpts);
};