CODEWITHSUNDEEP
mongodb
dev do
dev do

Reputation: 11

finding duplicates using map reduce from mongodb

I need to find the duplicates in a collection in mongo db which has around 20000 documents. The result should give me the key (on which I am grouping) and the count of times they are repeated only if the count is greater than 1. The below is not complete, however it is giving an error also when I run in mongo.exe shell :

db.runCommand({ mapreduce: users, 
 map : function Map() {
                emit(this.emailId, 1); 
}
reduce : function Reduce(key, vals) {
                return Array.sum(vals);

}
finalize : function Finalize(key, reduced) {
                return reduced
}
out : { inline : 1 }
});

SyntaxError: missing } after property list (shell):5

why is the above error coming? how can only get the ones with count greater than 1?

Upvotes: 1

Views: 1900

Answers (1)

dcrosta
dcrosta

Reputation: 26258

I'm not sure if that is an exact copy of the code you've entered, but it looks like you're missing commas between the fields in the object being passed to runCommand. Try:

db.runCommand({ mapreduce: users, 
    map : function Map() {
        emit(this.emailId, 1); 
    },
    reduce : function Reduce(key, vals) {
        return Array.sum(vals);
    },
    finalize : function Finalize(key, reduced) {
        return reduced
    },
    out : { inline : 1 }
});

Also note that even when using finalize, you can't actually remove entries from the outputted document (or collection) in a single-pass with Map-Reduce. However, whether you're using out: {inline: 1}, or out: "some_collection", it is pretty trivial to filter out results where the count is 1.

Upvotes: 1

Related Questions