CODEWITHSUNDEEP
ruby-on-railsmongodbmongoid
Gundam Meister
Gundam Meister

Reputation: 1495

How to upsert many in Mongo + Rails?

Everyday, I have a 500mb file that I need parse and save to the db.

When the db is clear, insert_many works for me.

Device.collection.insert_many(devices, ordered=false)

[ {id:"111", type:"aaa"},
  {id:"222", type:"bbb"},
  {id:"333", type:"ccc"} ]

However, some of the types in the array could change daily. And the array length could also change.

Question, how do I do a upsert_many with Mongoid? If id match, update if needed. If id not found, save the new document.

Thanks.

Upvotes: 1

Views: 1250

Answers (1)

D. SM
D. SM

Reputation: 14490

As far as I can tell the most efficient way of doing this is through the driver's bulk write API.

Use update_one with upsert: true: https://docs.mongodb.com/ruby-driver/master/tutorials/ruby-driver-bulk-operations/#update-one

Given your devices array, I would map each element to a bulk write operation which finds by the id in the array element and sets the attributes, with upsert: true. Then feed these bulk operations to Collection#bulk_write. You may need to split the entire devices array into chunks to stay under the 16 MB limit for BSON documents or for performance reasons.

Upvotes: 1

Related Questions