CODEWITHSUNDEEP
hadoopformathdfssqoop
Bohdan
Bohdan

Reputation: 17203

Can I use Sqoop to import data into RCFile format?

According to http://sqoop.apache.org/docs/1.4.0-incubating/SqoopUserGuide.html#id1764646

You can import data in one of two file formats: delimited text or SequenceFiles.

But what about RCFile?

Is it possible to use Sqoop to import data from Oracle DB into HDFS in RCFile format?

If yes, how to do it?

Upvotes: 1

Views: 1852

Answers (2)

Anoop Velluva
Anoop Velluva

Reputation: 329

Step 1: Create a ORC formatted table (base) in Hive.
    CREATE TABLE IF NOT EXISTS  tablename (hivecolumns)   STORED AS RCFILE

 Step 2 : Sqoop import to this RC table using HCatalog tool.  
     SQOOP IMPORT 
    --connect sourcedburl
    --username XXXX
    --password XXXX
     --table source_table
     --hcatalog-database hivedb
    --hcatalog-table tablename

[ HCatalog’s table abstraction presents users with a relational view of data in the Hadoop distributed file system (HDFS) and ensures that users need not worry about where or in what format their data is stored — RCFile format, text files, SequenceFiles, or ORC files.]

Upvotes: 0

Jarek Jarcec Cecho
Jarek Jarcec Cecho

Reputation: 1726

Sqoop is currently not supporting RC files. There is a jira SQOOP-640 to add this functionality.

Upvotes: 3

Related Questions