CODEWITHSUNDEEP
mongodbexport
malayladu
malayladu

Reputation: 11

Export each document of a mongodb collection into seperate json file

I have a mongoddb DB called 'CRM' and in that there is a collection "users".

There are 50,000+ users into "users" collection.

I want to export all users documents from "users" collection to separate .json file.

I ran following command which gives me all users into a single (users.json)json file as an array of json objects. 

mongoexport -d CRM -c users -o users.json --jsonArray

But, I want .JSON file for each users.

Any help would greatly be appreciated.

Upvotes: 1

Views: 2546

Answers (1)

Batandwa
Batandwa

Reputation: 550

jq helped me get this done.

The output file from Mongo is not a valid JSON file, rather individual lines of respectively valid JSON content. The steps I used were:

  • iterated through each line,
  • parse it to get the ID (you can use your own field here)
  • use the ID as the file name
  • write the line/document to a file

Here's the bash code I ultimately used:

# Make newlines the only separator.
IFS=$'\n'
# Disable globbing.
set -f          

# Iterate through the export file
for doc in $(cat < export.txt); do
  # $doc is a valid JSON document. Parse it to get the filename
  # based on the ID.
  export filename=`echo $doc | jq -r '._id'`.json

  # Write document to file.
  echo "Writing $filename"
  echo $doc > $filename
done

I learnt this from these Stack Exchange answers:

I'm just not sure how well this will work for 50, 000 records.

Upvotes: 1

Related Questions