CODEWITHSUNDEEP
sqoop
Somesh Tiwari
Somesh Tiwari

Reputation: 3

Create single file while using sqoop import with multiple mappers

I want to import data from Cobol-DB2 using sqoop import but my requirement is i want to use 4 mappers but it should create only one file in hdfs target directory. Is there any way to do this ?

Upvotes: 0

Views: 169

Answers (2)

Chema
Chema

Reputation: 2838

I think it's not possible with Sqoop, but you can merge a directory afterward with HDFS command as follow

-getmerge [-nl] <src> <localdst> :
  Get all the files in the directories that match the source file pattern and
  merge and sort them to only one file on local fs. <src> is kept.

as an example

hdfs dfs -getmerge -nl /user/dir/cobol /user/dir/cobol_merged

Upvotes: 0

Piyush Patel
Piyush Patel

Reputation: 1751

I don't think it is possible without using single mapper. If your target is to get single file in final database, then you can create intermediate staging table with multiple mappers and you can move data from staging table to final table to generate single file.

Upvotes: 1

Related Questions