CODEWITHSUNDEEP
javascriptarraysnode.jsmongodbmongoose
MattDionis
MattDionis

Reputation: 3616

How to purge duplicate objects from MongoDB

I have an array inside an object in my database that accumulates duplicate objects over time (a drawback of working with the unreliable Instagram API). I'm attempting to routinely purge this array of duplicates and replace it with the cleaned up array. I'm having a difficult time understanding why the below code run on my node server does not work. The trimArray function works perfectly, but the 'photos' array within my MongoDB object is never updated. 

// Takes in an array and outputs an array with only unique objects  
function trimArray(bloatedArray) {
    var seen = {};
    var trimmedArray = [];
    var len = bloatedArray.length;

    var j = 0;
    for(var i = 0; i < len; i++) {
        var imageLink = bloatedArray[i].link;
        var image = bloatedArray[i];
        if(seen[imageLink] !== 1) {
            seen[imageLink] = 1;
            trimmedArray[j++] = image;
        }
    }

    return trimmedArray;
}

Event.find( { $and: [{latitude: latitude}, {radius: distance}] },
    function(err,event){
        if (err) {

        } else {
            var array = event[0].photos;

            Event.update( { $and: [{latitude: latitude}, {radius: distance}] },
            { 'photos': trimArray(array) }
            );
        }
    }
);

Upvotes: 0

Views: 40

Answers (1)

Brandon Smith
Brandon Smith

Reputation: 1197

I think update will simply update existing records, not remove them. It looks for items in the returned array and updates them accordingly. If you want to use your trimArray, you'll have to empty the collection and then reinsert the trimArray results (terrible idea).

Instead, you should set your collection up correctly to not store duplicates in the first place. You'll want to set up a unique index, and then you'll never have to purge.

See the docs for details.

db.things.ensureIndex({'photos.make_some_key' : 1}, {unique : true, dropDups : true})

Upvotes: 1

Related Questions