CODEWITHSUNDEEP
hadoophiveflume
CK Chang
CK Chang

Reputation: 33

How to use flume creating a task to load data from HDFS to HIVE automatic by time?

I need to load data from hadoop to hive automatic, but I don't wanna setup an other service to do this thing. I already use flume to collect my logs...... So how should I do? Does flume can exec a command (a query for hive just like LOAD.....)?

Upvotes: 1

Views: 2474

Answers (2)

hadoopNerd
hadoopNerd

Reputation: 71

Sorry I am a little late to the party here, but I have actually put together a very complete example of how to do this with all the details exposed. Perhaps, it will help someone else

http://www.lopakalogic.com/articles/hadoop-articles/log-files-flume-hive/

Best of luck to you!

Upvotes: 2

Rags
Rags

Reputation: 1881

You can rather have Flume to store the extracted data under Hive's table directory (typically /user/hive/warehouse/Your_Table_Dierctory). Nothing more needs to be done.

Upvotes: 1

Related Questions