CODEWITHSUNDEEP
javascriptnode.jsmongodbmongoosedatabase
npclaudiu
npclaudiu

Reputation: 2441

What is the recommended way to drop indexes using Mongoose?

I need to create several deployment scripts like data migration and fixtures for a MongoDB database and I couldn't find enough information about how to drop indexes using Mongoose API. This is pretty straight-forward when using the official MongoDB API:

To delete all indexes on the specified collection:

db.collection.dropIndexes();

However, I would like to use Mongoose for this and I tried to use executeDbCommand adapted from this post, but with no success:

mongoose.connection.db.executeDbCommand({ dropIndexes: collectionName, index: '*' },
  function(err, result) { /* ... */ });

Should I use the official MongoDB API for Node.js or I just missed something in this approach?

Upvotes: 26

Views: 26053

Answers (5)

Nicholas Hamilton
Nicholas Hamilton

Reputation: 10506

This might not be the best place to post this, but I think its worth posting anyway.

I call model.syncIndexes() every time a model is defined/created against the db connection, this ensures the indexes are current and up-to-date with the schema, however as it has been highlighted online (example), this can create issues in distributed architectures, where multiple servers are attempting the same operation at the same time. This is particularly relevant if using something like the cluster library to spawn master/slave instances on multiple cores on the same machine, since they often boot up in close proximity to each other when the whole server is started.

In reference to the above 'codebarbarian' article, the issue is highlighted clearly when they state:

Mongoose does not call syncIndexes() for you, you're responsible for calling syncIndexes() on your own. There are several reasons for this, most notably that syncIndexes() doesn't do any sort of distributed locking. If you have multiple servers that call syncIndexes() when they start, you might get errors due to trying to drop an index that no longer exists.

So What I do is create a function which uses redis and redis redlock to gain a lease for some nominal period of time to prevent multiple workers (and indeed multiple workers in multiple servers) from attempting the same sync operation at the same time.

It also bypasses the whole thing unless it is the 'master' that is trying to perform the operation, I don't see any real point in delegating this job to any of the workers.

const cluster               = require('cluster');
const {logger}              = require("$/src/logger");
const { 
    redlock, 
    LockError
}                           = require("$/src/services/redis");
const mongoose              = require('mongoose');

// Check is mongoose model, 
// ref: https://stackoverflow.com/a/56815793/1834057
const isMongoModel = (obj) => {
    return obj.hasOwnProperty('schema') && obj.schema instanceof mongoose.Schema;
}

const syncIndexesWithRedlock = (model,duration=60000) => new Promise(resolve => {

    // Ensure the cluster is master
    if(!cluster.isMaster) 
        return resolve(false)

    // Now attempt to gain redlock and sync indexes
    try {

        // Typecheck
        if(!model || !isMongoModel(model))
            throw new Error('model argument is required and must be a mongoose model');

        if(isNaN(duration) || duration <= 0)
            throw new Error('duration argument is required, and must be positive numeric')

        // Extract name
        let name        = model.collection.collectionName;

        // Define the redlock resource
        let resource    = `syncIndexes/${name}`;

        // Coerce Duration to Integer
        // Not sure if this is strictly required, but wtf. 
        // Will ensure the duration is at least 1ms, given that duration <= 0 throws error above
        let redlockLeaseDuration = Math.ceil(duration);

        // Attempt to gain lock and sync indexes
        redlock.lock(resource,redlockLeaseDuration)
            .then(() => {
                // Sync Indexes
                model.syncIndexes();

                // Success
                resolve(true);
            })
            .catch(err => {
                
                // Report Lock Error
                if(err instanceof LockError){
                    logger.error(`Redlock LockError -- ${err.message}`);

                // Report Other Errors
                }else{
                    logger.error(err.message);
                }

                // Fail, Either LockError error or some other error
                return resolve(false);
            })

    // General Fail for whatever reason
    }catch(err){
        logger.error(err.message);
        return resolve(false);
    }
});

I wont go into setting up Redis connection, that is the subject of some other thread, but the point of this above code is to show how you can use syncIndexes() reliably and prevent issues with one thread dropping an index and another trying to drop the same index, or other distributed issues with attempting to modify indexes concurrently.

Upvotes: 1

Manoj Rana
Manoj Rana

Reputation: 3440

to drop a particular index you could use

db.users.dropIndex("your_index_name_here")

Upvotes: 0

J.Wolfe
J.Wolfe

Reputation: 798

If you want to maintain your indexes in your schema definitions with mongoose (you probably do if you're using mongoose), you can easily drop ones not in use anymore and create indexes that don't exist yet. You can just run a one off await YourModel.syncIndexes() on any models that you need to sync. It will create ones in the background with .ensureIndexes and drop any that no longer exist in your schema definition. You can look at the full docs here: https://mongoosejs.com/docs/api.html#model_Model.syncIndexes

Upvotes: 22

JohnnyHK
JohnnyHK

Reputation: 311945

To do this via the Mongoose model for the collection, you can call dropAllIndexes of the native collection:

MyModel.collection.dropAllIndexes(function (err, results) {
    // Handle errors
});

Update

dropAllIndexes is deprecated in the 2.x version of the native driver, so dropIndexes should be used instead:

MyModel.collection.dropIndexes(function (err, results) {
    // Handle errors
});

Upvotes: 41

Gates VP
Gates VP

Reputation: 45287

It looks like you're attempting to drop all of the indexes on a given collection.

According to the MongoDB Docs, this is the correct command.

... I tried to use executeDbCommand adapted from this post, but with no success:

To really help here, we need more details:

  • What failed? How did you measure "no success"?
  • Can you confirm 100% that the command ran? Did you output to the logs in the callback? Did you check the err variable?
  • Where are you creating indexes? Can you confirm that you're not re-creating them after dropping?
  • Have you tried the command while listing specific index names? Honestly, you should not be using "*". You should be deleting and creating very specific indexes.

Upvotes: 1

Related Questions