CODEWITHSUNDEEP
mongodbdockerdocker-composemongoimport
Jeff Voss
Jeff Voss

Reputation: 3695

Dockerized mongodb using mongoimport imports json array as item

I am attempting to import a json array of items into my mongodb instance on Docker. When trying without Docker, mongoimport responds with imported 7 documents but in Docker it responds imported 1 document

Docker Command:

CMD mongoimport --host mongodb --db reach-engine --collection domains --type json --file seed.json --jsonArray

Docker MongoDB:

{ "_id" : ObjectId("58dc3abd7ccf9a826b2a7d34"), "items" : [ { "id" : 1, "_id" : "item1", "type" : "alert", "title" : "hello.world", "email" : "[email protected]", "message" : "", "createdDate" : "date", "price" : "$9.00", "active" : true }, { "id" : 2, "_id" : "item2", "type" : "welcome.lol", "title" : "Item 2", "email" : "[email protected]", "message" : "lol", "createdDate" : "date", "price" : "$12.00", "active" : true }, { "id" : 3, "_id" : "item3", "type" : "message", "title" : "various.domain", "email" : "[email protected]", "message" : "lol", "createdDate" : "date", "price" : "$3.00", "active" : false }, { "id" : 4, "_id" : "item4", "type" : "message", "title" : "something.else", "message" : "", "createdDate" : "date", "price" : "$12.00", "active" : false }, { "id" : 5, "_id" : "item5", "type" : "update", "title" : "wow.lol", "email" : "[email protected]", "message" : "", "createdDate" : "date", "price" : "$12.00", "active" : false }, { "id" : 6, "_id" : "item6", "type" : "update", "title" : "domainname.net", "email" : "[email protected]", "message" : "cars", "createdDate" : "date", "price" : "$12.00", "active" : false }, { "id" : 7, "_id" : "item7", "type" : "update", "title" : "something.lol", "email" : "[email protected]", "message" : "", "createdDate" : "date", "price" : "$12.00", "active" : false } ] }

Local MongoDB:

{ "_id" : ObjectId("58dc21cf1ebe0a21fa2fcdd4"), "title" : "hello.world", "email" : "[email protected]", "message" : "", "createdDate" : "date", "price" : "$9.00", "active" : true }
{ "_id" : ObjectId("58dc21cf1ebe0a21fa2fcdd5"), "title" : "Item 2", "email" : "[email protected]", "message" : "lol", "createdDate" : "date", "price" : "$12.00", "active" : true }
{ "_id" : ObjectId("58dc21cf1ebe0a21fa2fcdd6"), "title" : "wow.lol", "email" : "[email protected]", "message" : "", "createdDate" : "date", "price" : "$12.00", "active" : false }
{ "_id" : ObjectId("58dc21cf1ebe0a21fa2fcdd7"), "title" : "domainname.net", "email" : "[email protected]", "message" : "cars", "createdDate" : "date", "price" : "$12.00", "active" : false }
{ "_id" : ObjectId("58dc21cf1ebe0a21fa2fcdd8"), "title" : "something.lol", "email" : "[email protected]", "message" : "", "createdDate" : "date", "price" : "$12.00", "active" : false }
{ "_id" : ObjectId("58dc21cf1ebe0a21fa2fcdd9"), "title" : "something.else", "message" : "", "createdDate" : "date", "price" : "$12.00", "active" : false }
{ "_id" : ObjectId("58dc21cf1ebe0a21fa2fcdda"), "title" : "various.domain", "email" : "[email protected]", "message" : "lol", "createdDate" : "date", "price" : "$3.00", "active" : false }
{ "_id" : ObjectId("58dc2b7c1ebe0a21fa2fcddb"), "title" : "hello.world", "email" : "[email protected]", "message" : "", "createdDate" : "date", "price" : "$9.00", "active" : true }
{ "_id" : ObjectId("58dc2b7c1ebe0a21fa2fcddc"), "title" : "Item 2", "email" : "[email protected]", "message" : "lol", "createdDate" : "date", "price" : "$12.00", "active" : true }
{ "_id" : ObjectId("58dc2b7c1ebe0a21fa2fcddd"), "title" : "various.domain", "email" : "[email protected]", "message" : "lol", "createdDate" : "date", "price" : "$3.00", "active" : false }
{ "_id" : ObjectId("58dc2b7c1ebe0a21fa2fcdde"), "title" : "something.else", "message" : "", "createdDate" : "date", "price" : "$12.00", "active" : false }
{ "_id" : ObjectId("58dc2b7c1ebe0a21fa2fcddf"), "title" : "wow.lol", "email" : "[email protected]", "message" : "", "createdDate" : "date", "price" : "$12.00", "active" : false }
{ "_id" : ObjectId("58dc2b7c1ebe0a21fa2fcde0"), "title" : "domainname.net", "email" : "[email protected]", "message" : "cars", "createdDate" : "date", "price" : "$12.00", "active" : false }
{ "_id" : ObjectId("58dc2b7c1ebe0a21fa2fcde1"), "title" : "something.lol", "email" : "[email protected]", "message" : "", "createdDate" : "date", "price" : "$12.00", "active" : false }
{ "_id" : ObjectId("58dc2d691ebe0a21fa2fcde2"), "title" : "various.domain", "email" : "[email protected]", "message" : "lol", "createdDate" : "date", "price" : "$3.00", "active" : false }
{ "_id" : ObjectId("58dc2d691ebe0a21fa2fcde3"), "title" : "Item 2", "email" : "[email protected]", "message" : "lol", "createdDate" : "date", "price" : "$12.00", "active" : true }
{ "_id" : ObjectId("58dc2d691ebe0a21fa2fcde4"), "title" : "hello.world", "email" : "[email protected]", "message" : "", "createdDate" : "date", "price" : "$9.00", "active" : true }
{ "_id" : ObjectId("58dc2d691ebe0a21fa2fcde5"), "title" : "wow.lol", "email" : "[email protected]", "message" : "", "createdDate" : "date", "price" : "$12.00", "active" : false }
{ "_id" : ObjectId("58dc2d691ebe0a21fa2fcde6"), "title" : "domainname.net", "email" : "[email protected]", "message" : "cars", "createdDate" : "date", "price" : "$12.00", "active" : false }
{ "_id" : ObjectId("58dc2d691ebe0a21fa2fcde7"), "title" : "something.lol", "email" : "[email protected]", "message" : "", "createdDate" : "date", "price" : "$12.00", "active" : false }

Docker CMD:

mongodb_1       | 2017-03-29T21:38:09.439+0000 I COMMAND  [conn1] command reach-engine.domains command: insert { insert: "domains", documents: [ { items: [ { id: 1, _id: "item1", type: "alert", title: "hello.world", email: "[email protected]", message: "", createdDate: "date", price: "$9.00", active: true }, { id: 2, _id: "item2", type: "welcome.lol", title: "Item 2", email: "[email protected]", message: "lol", createdDate: "date", price: "$12.00", active: true }, { id: 3, _id: "item3", type: "message", title: "various.domain", email: "[email protected]", message: "lol", createdDate: "date", price: "$3.00", active: false }, { id: 4, _id: "item4", type: "message", title: "something.else", message: "", createdDate: "date", price: "$12.00", active: false }, { id: 5, _id: "item5", type: "update", title: "wow.lol", email: "[email protected]", message: "", createdDate: "date", price: "$12.00", active: false }, { id: 6, _id: "item6", type: "update", title: "domainname.net", email: "[email protected]", message: "cars", createdDate: "date", price: "$12.00", active: false }, { id: 7, _id: "item7", type: "update", title: "something.lol", email: "[email protected]", message: "", createdDate: "date", price: "$12.00", active: false } ] } ], writeConcern: { getLastError: 1, w: 1 }, ordered: false } ninserted:1 keyUpdates:0 writeConflicts:0 numYields:0 reslen:40 locks:{ Global: { acquireCount: { r: 2, w: 2 } }, Database: { acquireCount: { w: 1, W: 1 } }, Collection: { acquireCount: { W: 1 } } } protocol:op_query 250ms

What could be causing this?

Upvotes: 5

Views: 1197

Answers (1)

s7vr
s7vr

Reputation: 75984

The docker mongo server behavior (possibly 3.x version) is correct i.e. inserting a single document with items as embedded array.

The local mongo behavior (possibly 2.x version) is because of JSON formatting issue.

I've followed below steps to reproduce the issue with the information provided in the post.

Consider sample1.json

{items:[{item:1},{item:2},{item:3}]}

Works exactly same in both 2.6 and 3.4 version.

Collection

{ "_id" : ObjectId("58e44c281a573a105fe10d47"), "items" : [ { "item" : 1 }, { "item" : 2 }, { "item" : 3 } ] }

Consider sample2.json ( has CRLF )

{"items":[{"item":1}, {"item":2}, {"item":3}]}

It works same as above for 3.4 version but works differently in 2.6.

It fails while inserting the first element but continue inserting the next two element as separate document in 2.6 version.

Command

mongoimport --db test --collection sample --type json --file sample2.json

Log

exception:BSON representation of supplied JSON is too large: code FailedToParse: FailedToParse: Bad characters in value: offset:21 of:{"items":[{"item":1}, 2017-04-04T21:03:13.869-0500 imported 2 objects encountered 1 error(s)

Collection

{ "_id" : ObjectId("58e4506169a04993a1697f29"), "item" : 2 }
{ "_id" : ObjectId("58e4506169a04993a1697f2a"), "item" : 3 }

Fix:

You have to update your json to below ( json formatting doesn't matter) to insert each element as a document and use --jsonArray flag. It works exactly the same in both 2.6 and 3.4.

Consider sample.json

[{"item":1}, {"item":2}, {"item":3}]

mongoimport --db test --collection sample --type json --file sample.json --jsonArray

Log

2017-04-04T21:19:34.018-0500 imported 3 objects

Collection

{ "_id" : ObjectId("58e454362a9c7ef0228df346"), "item" : 1 }
{ "_id" : ObjectId("58e454362a9c7ef0228df347"), "item" : 2 }
{ "_id" : ObjectId("58e454362a9c7ef0228df348"), "item" : 3 }

Upvotes: 4

Related Questions