CODEWITHSUNDEEP
mongodb
user112758
user112758

Reputation: 109

update local mongo dbs after copy it from remote server

I have copied a db from a remote mongodb server to my local machine. Now more data are added into the remote server and I want to update the local db from it. How should I achieve this? Thanks!

Upvotes: 0

Views: 973

Answers (2)

Pete Garafano
Pete Garafano

Reputation: 4913

Full disclosure, I work for MongoDB. Now that that is out of the way, there are essentially 2 main ways to do what you are asking:

  1. Do a full copy to get the "new" data
  2. Do an incremental copy to get the "new" data

Each one has advantages and disadvantages.

Full Copy

You have 4 options here

  1. Use the helper function db.copyDatabase
  2. Use SCP (or other file transfer protocol) to copy the underlying data files. If you are using WiredTiger you must copy all data. With MMAPv1, you can get away with copying just a database. There is no collection level resolution.
  3. Use mongodump to dump the entire database (or collection)
  4. Use a tool like MongoDB Cloud Manager to backup the remote server and restore to the local server. Cloud Manager even offers the ability to do automated restores using just the UI.

Incremental Copy

There are many options, here, I am going to list 2 that I am most familiar with

  1. Use mongodump in conjunction with a query to dump only the "latest" data. This requires a schema update on your part to track the timestamp the data was inserted or last updated.
  2. Use mongorestore in conjunction with mongodump to transfer just the oplog to the local server. This is outlined more or less here
  3. Use mongooplog to transfer the OpLog between the two servers.

Both options 2 and 3 will most likely require you to do some sort of namespace filtering to ensure you only get OpLog related to the specific collection or database you are concerned with moving.

My general recommendation is to use one of the following

  1. Full Copy with mongodump
  2. Using a tool like MongoDB Cloud Manager
  3. Full Copy with SCP (or other file transfer protocol)

These leave the least room for error and are generally quite robust for a variety of needs.

Upvotes: 1

profesor79
profesor79

Reputation: 9473

It depends of the data you have. If there is a timestamp field you could use mongo export with query parameter and ask for new documents since last copy.

What I will suggest is to create full data dump using mongodump and import data using mongorestore.

Upvotes: 0

Related Questions