Node and Mongoose - Not reconnecting if mongod was not running when first tried to connect

Question

Im using docker-composer and Im finding issues with execution order of services. The main issue happens when my express app tries to connect to mongod but this is not yet ready.

The issue can be reproduced easily by running first the nodejs application but not mongod (manually forcing this case).

My app uses mongoose and try to establish connection to mongod. Because mongod is not up and running, the app throws an error about it.

$ nodemon server/app.js 
24 Apr 21:42:05 - [nodemon] v1.7.0
24 Apr 21:42:05 - [nodemon] to restart at any time, enter `rs`
24 Apr 21:42:05 - [nodemon] watching: *.*
24 Apr 21:42:05 - [nodemon] starting `node server/app.js`
Listening on port 8000
disconnected
connection error: { [MongoError: connect ECONNREFUSED] name: 'MongoError', message: 'connect ECONNREFUSED' }

Starting mongod later seems to reconnect.

24 Apr 21:51:28 - [nodemon] v1.7.0
24 Apr 21:51:28 - [nodemon] to restart at any time, enter `rs`
24 Apr 21:51:28 - [nodemon] watching: *.*
24 Apr 21:51:28 - [nodemon] starting `node server/app.js`
Listening on port 8000
disconnected
connection error: { [MongoError: connect ECONNREFUSED] name: 'MongoError', message: 'connect ECONNREFUSED' }
connected
reconnected

Despite of that, operations that require access to mongo will not come through... neither error is shown

This is the code to connect to mongo using mongoose:

// Starting mongo
mongoose.connect(config.database, {
                                    server:{
                                            auto_reconnect:true,
                                            reconnectTries: 10,
                                            reconnectInterval: 5000,
                                    }
                                });

// Listening for connection
var mongo = {};
var db = mongoose.connection;
db.on('connected', console.error.bind(console, 'connected'));
db.on('error', console.error.bind(console, 'connection error:'));
db.on('close', console.error.bind(console, 'connection close.'));
db.once('open', function() {
    console.log("We are alive");
});
db.on('reconnected', function(){
    console.error('reconnected');   
});
db.on('disconnected', console.error.bind(console, 'disconnected'));

And here is the route that will try to get data from mongo but fail.

router.post('/auth', function(req, res){

    User.findOne({name: req.body.name})
        .then(function(user){

            if(!user)
            {
                res.status(401).send({ success: false, message: 'Authentication failed. User not found.' });
            }
            ...

How can I recover from running nodejs before mongo is ready?.

Answer 1

I had the same issue with Mongoose 5+. I was able to get this working by creating a retry function using set timeout.

const mongoose = require('mongoose');

const {
  MONGO_USERNAME,
  MONGO_PASSWORD,
  MONGO_HOSTNAME,
  MONGO_PORT,
  MONGO_DB,
  MONGO_DEBUG,
  MONGO_RECONNECT_TRIES,
  MONGO_RECONNECT_INTERVAL,
  MONGO_TIMEOUT_MS,
} = process.env;

if (MONGO_DEBUG) {
  console.log(`********* MongoDB DEBUG MODE *********`);
  mongoose.set('debug', true);
}

const DB_OPTIONS = {
  useNewUrlParser: true,
  reconnectTries: MONGO_RECONNECT_TRIES,
  reconnectInterval: MONGO_RECONNECT_INTERVAL,
  connectTimeoutMS: MONGO_TIMEOUT_MS,
};

const DB_URL = `mongodb://${MONGO_USERNAME}:${MONGO_PASSWORD}@${MONGO_HOSTNAME}:${MONGO_PORT}/${MONGO_DB}?authSource=admin`;

// Initialize conenction retry counter
let reconnectTriesAlready = 1;

// Connect to database with timeout and retry
const connectWithRetry = () => {
  mongoose.connect(DB_URL, DB_OPTIONS).then(() => {
    // Connected successfully
    console.log('********* MongoDB connected successfully *********');
    // Reset retry counter
    reconnectTriesAlready = 1;
  }).catch(err => {
    // Connection failed
    console.error(`********* ERROR: MongoDB connection failed ${err} *********`)
    // Compare retries made already to maximum retry count
    if (reconnectTriesAlready <= DB_OPTIONS.reconnectTries) {
      // Increment retry counter
      reconnectTriesAlready = reconnectTriesAlready + 1;
      // Reconnect retries made already has not exceeded maximum retry count
      console.log(`********* MongoDB connection retry after ${MONGO_RECONNECT_INTERVAL / 1000} seconds *********`)
      // Connection retry
      setTimeout(connectWithRetry, MONGO_RECONNECT_INTERVAL)
    } else {
      // Reconnect retries made already has exceeded maximum retry count
      console.error(`********* ERROR: MongoDB maximum connection retry attempts have been made already ${DB_OPTIONS.reconnectTries} stopping *********`)
    }
  })
}

connectWithRetry();

Answer 2

In my case, I created separate function only for mongoose connect method:

const connect = () => {
    mongoose.connect('mongodb://localhost:27017/myapp', {
        useNewUrlParser: true,
        reconnectTries: Number.MAX_VALUE,
        reconnectInterval: 500,
        poolSize: 10,
    });
};

I'm calling it at the same start. I also added Event Handler for error event:

mongoose.connection.on('error', (e) => {
    console.log('[MongoDB] Something went super wrong!', e);
    setTimeout(() => {
        connect();
    }, 10000);
});

If mongoose fails to connect because MongoDB is not running, error event handler is fired and setTimeout schedules "custom" reconnect.

Hope it helps.

Answer 3

How long does it take before mongod is ready? Because it seems like this is an edge case issue, where mongod might take a couple of seconds to get ready; and when mongoose is connected it serves requests as expected. Just trying to understand why the slight delay (probably a only a few seconds) is necessary to resolve?

But here is a solution anyway:

You could set up an express middleware to check if mongoose is ready and throw an error if not:

app.use(function(req,res,next){
    if (mongoose.Connection.STATES.connected === mongoose.connection.readyState){
        next();
    } else {
        res.status(503).send({success:false, message: 'DB not ready' });
    }
});

This should go before you inject your router.