CODEWITHSUNDEEP
mongodbcsvimport
Monyak
Monyak

Reputation: 38

MongoDB import CSV ignoring errors

Is it possible to import data from CSV into mongoDB using mongoimport, but ignoring any errors during parsing csv row? I have a csv file with about 200 millions of rows, and some of them are incorrectly formatted, and each iteration of fix/try takes a lot of time.

Upvotes: 1

Views: 3051

Answers (1)

Nilesh
Nilesh

Reputation: 2168

Use --parseGrace skipRow option.

mongoimport --type csv --parseGrace skipRow --file millionrecords.csv --headerline

--parseGrace grace

Default: stop

New in version 3.4.

Specifies how mongoimport handles type coercion failures when importing CSV or TSV files with --columnsHaveTypes.

--parseGrace has no effect when importing JSON documents.

  • autoCast : Assigns a type based on the value of the field. For example, if a field is defined as a double and the value for that field was "foo", mongoimport would make that field value a string type.
  • skipField : For the row being imported, mongoimport does not include the field whose type does not match the expected type.
  • skipRow : mongoimport does not import rows containing a value whose type does not match the expected type.
  • stop : mongoimport returns an error that ends the import.

Upvotes: 3

Related Questions