CODEWITHSUNDEEP
hadoopsqoop
Bhagwant Bhobe
Bhagwant Bhobe

Reputation: 79

Appending Data to hive Table using Sqoop

I am trying to append data to already existing Table in hive.Using the Following command first i import the table from MS-SQL Server to hive.

Sqoop Command:
sqoop import --connect "jdbc:sqlserver://XXX.XX.XX.XX;databaseName=mydatabase" --table "my_table" --where "Batch_Id > 100" --username myuser --password mypassword --hive-import

Now i want to append the data to same existing table in hive where "Batch_Id < 100" I am using the following Command:

sqoop import --connect "jdbc:sqlserver://XXX.XX.XX.XX;databaseName=mydatabase" --table "my_table" --where "Batch_Id < 100" --username myuser --password mypassword --append --hive-table my_table

This command however runs successfully also updates the HDFS data, but when u connect to hive shell and query the table, the records which are appended are not visible. Sqoop updated the Data on hdfs "/user/hduser/my_table" but the data on "/user/hive/warehouse/batch_dim" is not updated.

How can reslove this issue.

Regards, Bhagwant Bhobe

Upvotes: 2

Views: 3411

Answers (2)

Abhinay
Abhinay

Reputation: 635

Try using 

sqoop  import  --connect "jdbc:sqlserver://XXX.XX.XX.XX;databaseName=mydatabase"  
               --table "my_table" --where "Batch_Id < 100"  
               --username  myuser  --password mypassword   
               --hive-import --hive-table my_table

when you are using --hive-import DO NOT use --append parameter.

Upvotes: 1

salvyp
salvyp

Reputation: 23

The Sqoop command you're using (--import) is only for ingesting records into HDFS. You need to use the --hive-import flag to import records into Hive.

See http://sqoop.apache.org/docs/1.4.2/SqoopUserGuide.html#_importing_data_into_hive for more details and for additional import configuration options (you may want to change the document reference to your version of Sqoop, of course).

Upvotes: 0

Related Questions