CODEWITHSUNDEEP
hadoopsqoop
Rags
Rags

Reputation: 1881

Handling bad records during sqoop import or export

I looked at the options provided by sqoop export operation but could not find any options to handle bad records. For example once in a while it is possible that a character is present where a number is expected in a huge set of records. Is there a way to handle these scenarios in sqoop without failing the job and providing the bad records in a file.

Upvotes: 3

Views: 2226

Answers (1)

Jarek Jarcec Cecho
Jarek Jarcec Cecho

Reputation: 1726

Sqoop currently expects that the data to export is clean and do not provide facilities to handle corrupted data. You can use MR/Pig/Hive job to clean your data up, prior using Sqoop to export them.

Upvotes: 1

Related Questions