CODEWITHSUNDEEP
mongodbmeteormongodb-querydatabase-performance
Jack Wild
Jack Wild

Reputation: 2122

Meteor collection (MongoDB) update very poor performance

I'm having issues optimising update performance for a Meteor collection.

I take a document from one collection (CollectionA), modify it, cherry-pick some of the content, and then update a near duplicate of it in another collection (CollectionB). I also upsert in case new documents have been added to CollectionA

Everything happens in a few milliseconds, but the update can take anywhere from 10–30+ seconds depending on the size of the document being updated. The documents are typically around 30kb...

I have tried without Meteor.defer, with writeConcern : 0, and on both local and cloud replica set clusters. Have also tried using insert instead of update. Nothing makes a noticeable difference.

cronTask(){

    CollectionA.find({isPublished : true, "approval.status" : { $gte : 1 } }).forEach((doc)=>{

        let newDoc = {
            parentId : doc._id,
            slug : doc.slug, // these never change, only the content array changes...
            title : doc.title,
            description: doc.description,
            tags : doc.tags,
            category : doc.category,
            createdAt : new Date(),
            content: [...,...,...] // the content of the new document is cherrypicked from the parents before saving
        }

        while(doc.content.length) {
            // cherry-picking and pushing to newDoc.content
            // super quick, a couple of MS
        }

        Meteor.defer(()=>{
            CollectionB.update({parentId : doc._id}, {$set : newDoc}, {upsert : true}, (err,res)=>{
                if (!err) {
                    console.log(`Saved child for ${doc.title}`);
                } else {
                    console.log(`Error saving child for ${doc.title}: ${err}`);
                }
            });
        });

    });
}

Upvotes: 0

Views: 916

Answers (1)

Jack Wild
Jack Wild

Reputation: 2122

Turned out that the issue was not in fact the update, it was schema validation (using https://github.com/aldeed/meteor-simple-schema).

I disabled schema checking for the objects in the array (the method is on the server so it's safe to not validate in this case). Now it takes <1ms to update all 30 documents.

Not sure why the schema validation was so slow.

Upvotes: 1

Related Questions