user2297081
user2297081

Reputation: 56

How do i import json file that was exported from parse.com into mongodb?

I exported our collections from parse.com and they are bunch of json files ranging from few hundred megs to few gigabytes. How would I import them onto a local mongo database?

I tried running

mongoimport -d dbname -c collectionname --file file.json --jsonArray

but that did not work because i got an error saying jsonArray too large.

Any help would be appreciated.

Upvotes: 3

Views: 1954

Answers (4)

Tyler Brock
Tyler Brock

Reputation: 30136

Both of the answers here are incorrect as mongoimport handles this use case just fine. MongoDB has no limit (aside from those of your own computer) as to how much data can be loaded into a database.

It is true that document size is indeed limited to 16mb by MongoDB BUT the Parse platform limits document size to 128kb so there is no chance that a even the largest Parse document(object) will exceed MongoDB's 16mb constraint.

Now, to import data from exported from Parse into MongoDB (I've succesfully done this with a multi GB export) I was able to use the --jsonArray option by removing the enclosing result: { } that Parse's export tool puts around the JSON objects. I also put all the options before the filename:

mongoimport -d dbname -c collectionname --jsonArray file.json`

You can easily strip the enclosing result object with jq as mentioned in the comments using the following command if you have a lot of collections to process:

jq .results <collectionFromParse.json>

Upvotes: 7

Chris Conover
Chris Conover

Reputation: 9039

Credit to @Tyler, but as a notes-to-self entry, if you want to replicate a Parse data set on a machine for use with Parse-Server, you also need to map Parse's objectId indices to Mongo's _id indices. There are many ways; I just piped it through sed, as below:

# brew install jq # (if on a Mac)
sed -e 's/\"objectId\":/\"_id\":/g' <collection.json> \
    | jq .results | mongoimport -d <DatabaseName>

Upvotes: 0

Randall Hunt
Randall Hunt

Reputation: 12582

I believe the issue is that jsonArray is limited to 16mb total.

You can still use mongoimport but you'll need to make sure each document is on a single line and that the results key is removed.

You can do this with sed and stream the results into mongoimport.

Upvotes: 0

wdberkeley
wdberkeley

Reputation: 11671

The --jsonArray option restricts the import to 16MB and expects the import of multiple MongoDB documents in one json array. In general, documents in MongoDB have a 16MB size limit. You'll have to modify the json files and design your schema so each document is < 16MB.

Upvotes: -1

Related Questions