CODEWITHSUNDEEP
mongodbbackupdatabase-backups
JOsip
JOsip

Reputation: 13

What are important mongo data files for backup

If I want to backup database by copying raw files. What files do I need to copy ? only db-name.ns, db-name.0, db-name.1.... or whole folder (local.ns.., journal). I'm running replica set. I understand procedure for locking hidden secondary node and then copying files to new location. But I'm wondering do I need to copy whole folder or just some files.

Thx

Upvotes: 1

Views: 2481

Answers (3)

Markus W Mahlberg
Markus W Mahlberg

Reputation: 20703

Simple answer: All of them. As obvious as it might sound. And here is why:

  1. If you don't copy a namespaces file, your database will most likely not work.
  2. When not copying all datafiles, some of your data is missing and your indices will point to void locations. The database in question might work (minus the data stored in the missing data file), but I would not bet on that – and since the data was important enough to create a backup in the first place, you don't want this to happen, do you?
  3. Config, admin and local databases are vitally necessary for their respective features – and since you used the feature, you probably want to use it after a restore, too.

How do I backup all files?

The best solution save for MMS backup I have found so far is to create LVM snapshots of the filesystem the MongoDB data resides on. In order for tis to work, the journal needs to be included. Usually, you don't need a dedicated backup node for this approach. It is a bit complicated to set up, though.

Preparing LVM backups

Let's assume you have your data in the default data directory /data/db and you have not changed any paths. Then you would mount a logical volume to /data/db and use this to hold the data. Assuming that you don't have anything like this, here is a step by step guide:

  1. Create a logical volume big enough to hold your data. I will call that one /dev/VolGroup/LogVol1 from now on. Make sure that you only use about 80% of the available disk space in the volume group for creating the logical volume.

  2. Create a filesystem on the logical volume. I prefer XFS, so we create an xfs filesystem on /dev/VolGroup/LogVol1:

    mkfs.xfs /dev/VolGroup/LogVol1

  3. Mount the newly created filesystem on /mnt

    mount /dev/VolGroup/LogVol1 /mnt

  4. Shut down mongod:

    killall mongod

    (Note that the upstart scripts sometimes have problems shutting down mongod, and this command gracefully stops mongod anyway).

  5. Copy the datafiles from /data/dbto /mntby issuing

    cp -a /data/db/* /mnt

  6. Adjust your /etc/fstab so that the logical volume gets mounted on reboot:

    # The noatime parameter increases io speed of mongod significantly
/dev/VolGroup/LogVol1   /data/db   xfs   defaults,noatime    0    1

  7. Umount the logical volume from it's current outpoint and remount it on the correct one:

    cd && umount /mnt/ && mount /data/db

  8. Restart mongod

Creating a backup

Creating a backup now becomes as easy as

  1. Create a snapshot:

    lvcreate -l100%FREE -s -n mongo_backup /dev/VolGroup/LogVol1

  2. Mount the snapshot:

    mount /dev/VolGroup/mongo_backup /mnt

  3. Copy it somewhere. The reason we need to do this is that the snapshot can only be held up until the changes to the data files do not exceed the space in the volume group you did not allocate during preparation. For example, if you have a 100GB disk and you allocated 80GB for /dev/VolGroup/LogVol1, the snapshot size would be 20GB. While the changes on the filesystem from the point you took the snapshot are less than 20GB, everything runs fine. After that, the filesystem will refuse to take any changes. So you aren't in a hurry, but you should definitely move the data to an offsite location, an FTP server or whatever you deem appropriate. Note that compressing the datafiles can take quite long and you might run out of "change space" before finishing that. Personally, I like to have a slower HDD as a temporary place to store the backup, doing all other operations on the HDD. So my copy command looks like

    cp -a /mnt/* /home/mongobackup/backups

when the HDD is mounted on /home/mongobackup.

  1. Destroy the snapshot:

    umount /mnt && lvremove /dev/VolGroup/mongo_backup

    The space allocated for the snapshot is released and the restrictions to the amount of changes to the filesystem are removed.

Upvotes: 3

vmr
vmr

Reputation: 1935

The best solution to backup data on MongoDB would be to use Mongo monitoring Service(MMS). All other solutions including copying files manually, mongodump, mongoexport are way behind MMS.

Upvotes: 0

Dave Gordon
Dave Gordon

Reputation: 1835

Whole db-Data folder + where ever you have your logs and journalling

Upvotes: 0

Related Questions