CODEWITHSUNDEEP
mongodbexport
aboutstudy
aboutstudy

Reputation: 5077

How to export all collections in MongoDB?

I want to export all collections in MongoDB by the command: 

mongoexport -d dbname -o Mongo.json

The result is:
No collection specified!

The manual says, if you don't specify a collection, all collections will be exported.
However, why doesn't this work?

http://docs.mongodb.org/manual/reference/mongoexport/#cmdoption-mongoexport--collection

My MongoDB version is 2.0.6.

Upvotes: 477

Views: 529855

Answers (30)

StackOverflower
StackOverflower

Reputation: 437

My fellow developer @Mentor Reka is correct but i want to spoon feed it to new people to mongodb.

Note: This is for mongodb local host.

  1. Download MongoDB Command Line Database Tools Download
  2. Extract the zip file somewhere and add the .../bin path into your system environment
  3. open you terminal and write this mongodump --host localhost --port 27017

Node: wherever your .../bin path is it create a file name dump and exports all your data there. If you want to export in specific path use this: mongodump --host localhost --port 27017 --out /path/to/your/directory

Upvotes: 1

Vahid Najafi
Vahid Najafi

Reputation: 5233

For local and remote dump and restore:

For Local

Local dump

mongodump -d mydb -o ./mongo-backup

Local restore

mongorestore -d mydb ./mongo-backup/mydb

For remote

Remote dump

mongodump --uri "mongodb+srv://Admin:[email protected]/mytestdb" -o ./mongo-backup

Remote restore

mongorestore --uri "mongodb+srv://Admin:[email protected]/mytestdb" ./mongo-backup/mytestdb

Update:

If you're using mongo 4.0 you may encounter a snapshot error, Then you can run with this argument: --forceTableScan. See here for more information. The error is something like this:

mongodump error reading collection: BSON field 'FindCommandRequest.snapshot' is an unknown field.

Upvotes: 87

samran
samran

Reputation: 723

Export The whole Mongo DB just by this commnad

mongoexport –db database_name –collection collection_name –out path_or_filename.json

Upvotes: -2

Nayem Jaman Tusher
Nayem Jaman Tusher

Reputation: 1078

This is the simplest technique to achieve your aim.

mongodump -d db_name -o path/filename.json

Upvotes: 3

vishal chaudhari
vishal chaudhari

Reputation: 21

First, of Start the Mongo DB - for that go to the path as ->

C:\Program Files\MongoDB\Server\3.2\bin and click on the mongod.exe file to start MongoDB server.

Command in Windows to Export

  • Command to export MongoDB database in Windows from "remote-server" to the local machine in directory C:/Users/Desktop/temp-folder from the remote server with the internal IP address and port.
C:\> mongodump --host remote_ip_address:27017 --db <db-name> -o C:/Users/Desktop/temp-folder

Command in Windows to Import

  • Command to import MongoDB database in Windows to "remote-server" from local machine directory C:/Users/Desktop/temp-folder/db-dir
C:\> mongorestore --host=ip --port=27017 -d <db-name> C:/Users/Desktop/temp-folder/db-dir

Upvotes: 2

Ravichandran K
Ravichandran K

Reputation: 386

Please let us know where you have installed your Mongo DB? (either in Ubuntu or in Windows)

  • For Windows:

  1. Before exporting you must connect to your Mongo DB in cmd prompt and make sure that you are able to connect to your local host.

  2. Now open a new cmd prompt and execute the below command,

    mongodump --db database name --out path to save

    eg: mongodump --db mydb --out c:\TEMP\op.json

  3. Visit https://www.youtube.com/watch?v=hOCp3Jv6yKo for more details.

  • For Ubuntu:

  1. Login to your terminal where Mongo DB is installed and make sure you are able to connect to your Mongo DB.

  2. Now open a new terminal and execute the below command,

    mongodump -d database name -o file name to save

    eg: mongodump -d mydb -o output.json

  3. Visit https://www.youtube.com/watch?v=5Fwd2ZB86gg for more details.

Upvotes: 26

Usman
Usman

Reputation: 2577

To export all collections:

mongodump -d database_name -o directory_to_store_dumps

To restore them:

mongorestore -d database_name directory_backup_where_mongodb_tobe_restored

Upvotes: 47

Joe
Joe

Reputation: 29

This is the bash script I used to achieve the result. The script is generalized to 4 inputs (host url, database, username, and password), so it can be used on any mongo database.

dburl=$1
username=$3
password=$4
db=$2

mongoAccess=mongodb+srv://$username:$password@$dburl/$db

Collections=$(mongo $mongoAccess --quiet --eval "db.getCollectionNames()" | sed 's/,/ /g' | tail +6)

#echo $Collections

for col in $Collections
do
    if [ "$col" = "[" ] || [ "$col" = "]" ]
    then
        continue
    else
        echo "Exporting $col"
        mongoexport --uri $mongoAccess --collection=$col --type json --out output-$col.json
    fi
    
done

Upvotes: 0

Krishnadas PC
Krishnadas PC

Reputation: 6519

Some of the options are now deprecated, in version 4.4.5 here is how I have done it

mongodump --archive="my-local-db" --db=my


mongorestore --archive="my-local-db" --nsFrom='my.*' --nsTo='mynew.*'

Read more about restore here: https://docs.mongodb.com/database-tools/mongorestore/

Upvotes: 3

vanduc1102
vanduc1102

Reputation: 6245

If you have this issue: Failed: can't create session: could not connect to server: connection() : auth error: sasl conversation error: unable to authenticate using mechanism "SCRAM-SHA-1": (AuthenticationFailed) Authentication failed.

then add --authenticationDatabase admin

eg:

mongodump -h 192.168.20.30:27018 --authenticationDatabase admin -u dbAdmin -p dbPassword -d dbName -o path/to/folder

Upvotes: 3

Coconut
Coconut

Reputation: 2222

If you're dealing with remote databases you can try these commands given that you don't mind the output being BSON

1. Dump out as a gzip archive

mongodump --uri="mongodb://YOUR_USER_ID:YOUR_PASSWORD@YOUR_HOST_IP/YOUR_DB_NAME" --gzip --archive > YOUR_FILE_NAME

2. Restore (Copy a database from one to another)

mongorestore --uri="mongodb://$targetUser:$targetPwd@$targetHost/$targetDb" --nsFrom="$sourceDb.*" --nsTo="$targetDb.*" --gzip --archive

Upvotes: 11

Vinh The Nguyen
Vinh The Nguyen

Reputation: 189

I dump all collection on robo3t. I run the command below on vagrant/homestead. It's work for me

mongodump --host localhost --port 27017 --db db_name --out db_path

Upvotes: 2

Vlad Hrona
Vlad Hrona

Reputation: 427

Even in mongo version 4 there is no way to export all collections at once. Export the specified collection to the specified output file from a local MongoDB instance running on port 27017 you can do with the following command:

.\mongoexport.exe --db=xstaging --collection=products --out=c:/xstaging.products.json

Upvotes: 0

Binara Medawatta
Binara Medawatta

Reputation: 532

You can do it using the mongodump command

Step 1 : Open command prompt

Step 2 : go to bin folder of your mongoDB installation (C:\Program Files\MongoDB\Server\4.0\bin)

Step 3 : then execute the following command

mongodump -d your_db_name -o destination_path

your_db_name = test

destination_path = C:\Users\HP\Desktop

Exported files will be created in destination_path\your_db_name folder (in this example C:\Users\HP\Desktop\test)

References : o7planning

Upvotes: 14

Mentor Reka
Mentor Reka

Reputation: 9397

For lazy people, use mongodump, it's faster:

mongodump -d <database_name> -o <directory_backup>

And to "restore/import" it (from directory_backup/dump/):

mongorestore -d <database_name> <directory_backup>

This way, you don't need to deal with all collections individually. Just specify the database.

Note that I would recommend against using mongodump/mongorestore for big data storages. It is very slow and once you get past 10/20GB of data it can take hours to restore.

Upvotes: 912

sudheer nunna
sudheer nunna

Reputation: 1719

For dump, your DB fallow the below CMD

   mongodump -d <your d name> -o <dump path>
Ex:mongodump -d qualetics -o D:\dbpackup\qualetics

Upvotes: 0

Ryan Taylor
Ryan Taylor

Reputation: 13415

There are multiple options depending on what you want to do

1) If you want to export your database to another mongo database, you should use mongodump. This creates a folder of BSON files which have metadata that JSON wouldn't have.

mongodump
mongorestore --host mongodb1.example.net --port 37017 dump/

2) If you want to export your database into JSON you can use mongoexport except you have to do it one collection at a time (this is by design). However I think it's easiest to export the entire database with mongodump and then convert to JSON.

# -d is a valid option for both mongorestore and mongodump

mongodump -d <DATABASE_NAME>
for file in dump/*/*.bson; do bsondump $file > $file.json; done

Upvotes: 1

Anup_Tripathi
Anup_Tripathi

Reputation: 2817

In case you want to connect a remote mongoDB server like mongolab.com, you should pass connection credentials eg.

mongoexport -h id.mongolab.com:60599 -u username -p password -d mydb -c mycollection -o mybackup.json

Upvotes: 12

manoj tiwari
manoj tiwari

Reputation: 89

#mongodump using sh script 
#!/bin/bash
TIMESTAMP=`date +%F-%H%M`
APP_NAME="folder_name"
BACKUPS_DIR="/xxxx/tst_file_bcup/$APP_NAME"
BACKUP_NAME="$APP_NAME-$TIMESTAMP"
/usr/bin/mongodump -h 127.0.0.1 -d <dbname> -o $BACKUPS_DIR/$APP_NAME/$BACKUP_NAME
tar -zcvf $BACKUPS_DIR/$BACKUP_NAME.tgz $BACKUPS_DIR/$APP_NAME/$BACKUP_NAME
rm -rf /home/wowza_analytics_bcup/wowza_analytics/wowza_analytics
### 7 days old backup delete automaticaly using given command

find /home/wowza_analytics_bcup/wowza_analytics/ -mindepth 1 -mtime +7 -delete

Upvotes: 2

AnoopGoudar
AnoopGoudar

Reputation: 924

Follow the steps below to create a mongodump from the server and import it another server/local machine which has a username and a password

1. mongodump -d dbname -o dumpname -u username -p password
2. scp -r user@remote:~/location/of/dumpname ./
3. mongorestore -d dbname dumpname/dbname/ -u username -p password

Upvotes: 38

kumar31rajesh
kumar31rajesh

Reputation: 71

you can create zip file by using following command .It will create zip file of database {dbname} provided.You can later import the following zip file in you mongo DB. 

Window filepath=C:\Users\Username\mongo 

mongodump --archive={filepath}\+{filename}.gz --gzip --db {dbname}

Upvotes: 6

karthikdivi
karthikdivi

Reputation: 3633

Previous answers explained it well, I am adding my answer to help in case you are dealing with a remote password protected database

mongodump --host xx.xxx.xx.xx --port 27017 --db your_db_name --username your_user_name --password your_password --out /target/folder/path

Upvotes: 18

Ravish
Ravish

Reputation: 2421

if you want to use mongoexport and mongoimport to export/import each collection from database, I think this utility can be helpful for you. I've used similar utility couple of times;

LOADING=false

usage()
{
    cat << EOF
    usage: $0 [options] dbname

    OPTIONS:
        -h      Show this help.
        -l      Load instead of export
        -u      Mongo username
        -p      Mongo password
        -H      Mongo host string (ex. localhost:27017)
EOF
}

while getopts "hlu:p:H:" opt; do
    MAXOPTIND=$OPTIND

    case $opt in 
        h)
            usage
            exit
            ;;
        l)
            LOADING=true
            ;;
        u)
            USERNAME="$OPTARG"
            ;;
        p) 
            PASSWORD="$OPTARG"
            ;;
        H)
            HOST="$OPTARG"
            ;;
        \?)
            echo "Invalid option $opt"
            exit 1
            ;;
    esac
done

shift $(($MAXOPTIND-1))

if [ -z "$1" ]; then
    echo "Usage: export-mongo [opts] <dbname>"
    exit 1
fi

DB="$1"
if [ -z "$HOST" ]; then
    CONN="localhost:27017/$DB"
else
    CONN="$HOST/$DB"
fi

ARGS=""
if [ -n "$USERNAME" ]; then
    ARGS="-u $USERNAME"
fi
if [ -n "$PASSWORD" ]; then
    ARGS="$ARGS -p $PASSWORD"
fi

echo "*************************** Mongo Export ************************"
echo "**** Host:      $HOST"
echo "**** Database:  $DB"
echo "**** Username:  $USERNAME"
echo "**** Password:  $PASSWORD"
echo "**** Loading:   $LOADING"
echo "*****************************************************************"

if $LOADING ; then
    echo "Loading into $CONN"
    tar -xzf $DB.tar.gz
    pushd $DB >/dev/null

    for path in *.json; do
        collection=${path%.json}
        echo "Loading into $DB/$collection from $path"
        mongoimport $ARGS -d $DB -c $collection $path
    done

    popd >/dev/null
    rm -rf $DB
else
    DATABASE_COLLECTIONS=$(mongo $CONN $ARGS --quiet --eval 'db.getCollectionNames()' | sed 's/,/ /g')

    mkdir /tmp/$DB
    pushd /tmp/$DB 2>/dev/null

    for collection in $DATABASE_COLLECTIONS; do
        mongoexport --host $HOST -u $USERNAME -p $PASSWORD -db $DB -c $collection --jsonArray -o $collection.json >/dev/null
    done

    pushd /tmp 2>/dev/null
    tar -czf "$DB.tar.gz" $DB 2>/dev/null
    popd 2>/dev/null
    popd 2>/dev/null
    mv /tmp/$DB.tar.gz ./ 2>/dev/null
    rm -rf /tmp/$DB 2>/dev/null
fi

Upvotes: 3

Tom Boutell
Tom Boutell

Reputation: 7572

I realize that this is quite an old question and that mongodump/mongorestore is clearly the right way if you want a 100% faithful result, including indexes.

However, I needed a quick and dirty solution that would likely be forwards and backwards compatible between old and new versions of MongoDB, provided there's nothing especially wacky going on. And for that I wanted the answer to the original question.

There are other acceptable solutions above, but this Unix pipeline is relatively short and sweet:

mongo --quiet mydatabase --eval "db.getCollectionNames().join('\n')" | \
grep -v system.indexes | \
xargs -L 1 -I {} mongoexport -d mydatabase -c {} --out {}.json

This produces an appropriately named .json file for each collection.

Note that the database name ("mydatabase") appears twice. I'm assuming the database is local and you don't need to pass credentials but it's easy to do that with both mongo and mongoexport.

Note that I'm using grep -v to discard system.indexes, because I don't want an older version of MongoDB to try to interpret a system collection from a newer one. Instead I'm allowing my application to make its usual ensureIndex calls to recreate the indexes.

Upvotes: 17

Mani
Mani

Reputation: 57

  1. Open the Connection
  2. Start the server
  3. open new Command prompt

Export:

mongo/bin> mongoexport -d webmitta -c domain -o domain-k.json

Import:

mongoimport -d dbname -c newCollecionname --file domain-k.json

Where 

webmitta(db name)
domain(Collection Name)
domain-k.json(output file name)

Upvotes: 0

codecowboy
codecowboy

Reputation: 10095

Here's what worked for me when restoring an exported database:

mongorestore -d 0 ./0 --drop

where ./contained the exported bson files. Note that the --drop will overwrite existing data.

Upvotes: 3

Rondo
Rondo

Reputation: 3711

If you want to dump all collections in all databases (which is an expansive interpretation of the original questioner's intent) then use

mongodump

All the databases and collections will be created in a directory called 'dump' in the 'current' location

Upvotes: 7

Garreth McDaid
Garreth McDaid

Reputation: 2613

If you want to backup all the dbs on the server, without having the worry about that the dbs are called, use the following shell script:

#!/bin/sh

md=`which mongodump`
pidof=`which pidof`
mdi=`$pidof mongod`
dir='/var/backup/mongo'

if [ ! -z "$mdi" ]
   then
        if [ ! -d "$dir" ]
           then
               mkdir -p $dir
           fi
        $md --out $dir >/dev/null 2>&1
   fi

This uses the mongodump utility, which will backup all DBs if none is specified.

You can put this in your cronjob, and it will only run if the mongod process is running. It will also create the backup directory if none exists.

Each DB backup is written to an individual directory, so you can restore individual DBs from the global dump.

Upvotes: 2

Mister P
Mister P

Reputation: 1273

I found after trying lots of convoluted examples that very simple approach worked for me.

I just wanted to take a dump of a db from local and import it on a remote instance:

on the local machine:

mongodump -d databasename

then I scp'd my dump to my server machine:

scp -r dump [email protected]:~

then from the parent dir of the dump simply:

mongorestore

and that imported the database.

assuming mongodb service is running of course.

Upvotes: 9

Michael
Michael

Reputation: 22937

If you want, you can export all collections to csv without specifying --fields (will export all fields).

From http://drzon.net/export-mongodb-collections-to-csv-without-specifying-fields/ run this bash script

OIFS=$IFS;
IFS=",";

# fill in your details here
dbname=DBNAME
user=USERNAME
pass=PASSWORD
host=HOSTNAME:PORT

# first get all collections in the database
collections=`mongo "$host/$dbname" -u $user -p $pass --eval "rs.slaveOk();db.getCollectionNames();"`;
collections=`mongo $dbname --eval "rs.slaveOk();db.getCollectionNames();"`;
collectionArray=($collections);

# for each collection
for ((i=0; i<${#collectionArray[@]}; ++i));
do
    echo 'exporting collection' ${collectionArray[$i]}
    # get comma separated list of keys. do this by peeking into the first document in the collection and get his set of keys
    keys=`mongo "$host/$dbname" -u $user -p $pass --eval "rs.slaveOk();var keys = []; for(var key in db.${collectionArray[$i]}.find().sort({_id: -1}).limit(1)[0]) { keys.push(key); }; keys;" --quiet`;
    # now use mongoexport with the set of keys to export the collection to csv
    mongoexport --host $host -u $user -p $pass -d $dbname -c ${collectionArray[$i]} --fields "$keys" --csv --out $dbname.${collectionArray[$i]}.csv;
done

IFS=$OIFS;

Upvotes: 7

Related Questions