AKG
AKG

Reputation: 618

Moving partial data across hadoop instances

I have to copy a certain chunk of data from one hadoop cluster to another. I wrote a hive query which dumps the data into hdfs. After copying the file to the destination cluster, I tried to load the data using the command "load data inpath '/a.txt' into table data". I got the following error message

Failed with exception Wrong file format. Please check the file's format. FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask

I had dumped the data as a sequence file. Can anybody let me know what am I missing here ?

Upvotes: 1

Views: 1263

Answers (1)

Tariq
Tariq

Reputation: 34184

You should use STORED AS SEQUENCEFILE while creating the table if you want to store sequence file in the table. And you have written that you have dumped data as Sequence file but your file name is a.txt. I didn't get that.

If you want to load a text file into a table that expects Sequence file as the data source you could do one thing. First create a normal table and load the text file into this table. Then do :

insert into table seq_table select * from text_table;

Upvotes: 2

Related Questions