Fastest Way to (1) not insert duplicate entry (2) consolidate duplicates in Mongo DB?

Question

(1) What's the fastest way to check if an item I'm about to "insert" into a MongoDB collection is unique (and if so not insert it)

(2) For an existing database, what's the fastest way to look at all the entries and remove duplicates but keep one copy i.e. like a "set" function: {a,b,c,a,a,b} -> {a,b,c}

• I am aware that technically speaking each entry is unique, since they get a unique ObjectID
• You may assume the entries are completely flat key:value lists
• Solutions with indexing are fine
• I prefer Python code (i.e. mongo python API) if possible

Thanks!

Answer 1

(1) Create a unique index on the related columns and catch the error upon insertion time