CODEWITHSUNDEEP
mongodbdatabasenosql
Jack
Jack

Reputation: 3943

MongoDB: How to merge two collections/databases together into one?

I have two databases named: DB_A and DB_B.

Each database has one collection with same name called store.

Both collections have lots lots of documents that have exactly the same structure { key:" key1", value: "value1" }, etc.

Actually, I was supposed to only create DB_A and insert all documents into DB_A. But later when I did my second round of inserting, I made a mistake by typing the wrong name as the database name.

So now, each database has a size of 32GB, I wish to merge two databases.

One problem/constraint is that the free space available now is only 15GB, so I can't just copy all things from DB_B to DB_A.

I am wondering if I can perform some kind of "move" to merge the two databases? I prefer the most efficient way as simply reinserting 32GB into DB_A will take quite a time.

Upvotes: 13

Views: 18744

Answers (2)

Xavier Guihot
Xavier Guihot

Reputation: 61736

Starting Mongo 4.2, the new aggregation stage $merge can be used to merge the content of a collection in another collection in another database: 

// > use db1
// > db.collection.find()
//   { "_id" : 1, "key" : "a", "value" : "b" }
//   { "_id" : 2, "key" : "c", "value" : "d" }
//   { "_id" : 3, "key" : "a", "value" : "b" }
// > use db2
// > db.collection.find()
//   { "_id" : 1, "key" : "e", "value" : "f" }
//   { "_id" : 4, "key" : "a", "value" : "b" }
// > use db1
db.collection.aggregate([
  { $merge: { into: { db: "db2", coll: "coll" } } }
])
// > use db2
// > db.collection.find()
//   { "_id" : 1, "key" : "a", "value" : "b" }
//   { "_id" : 2, "key" : "c", "value" : "d" }
//   { "_id" : 3, "key" : "a", "value" : "b" }
//   { "_id" : 4, "key" : "a", "value" : "b" }

By default, when the target and the source collections contain a document with the same _id, $merge will replace the document from the target collection with the document from the source collection. In order to customise this behaviour, check $merge's whenMatched parameter.

Upvotes: 6

Sergio Tulentsev
Sergio Tulentsev

Reputation: 230521

I think the easiest (and maybe the only) way is to write a script that merges the two databases document after document.

  1. Get first document from DB_B.
  2. Insert it into DB_A if needed.
  3. Delete it from DB_B.
  4. Repeat until done.

Instead of deleting documents from source db (DB_B), you may want to just read documents in batches. This should be more performant, but slightly more difficult to code (especially if you never done such a thing).

Upvotes: 9

Related Questions