CODEWITHSUNDEEP
hadoophdfsflume-ng
wazza
wazza

Reputation: 800

Flume to write logs to hdfs if there is any change in input file

I am new to Apache Flume and I have an usecase in which I have a input file "sample.txt" which is updated periodically and it contains the following data initially

Name   Age    Sex   Address
xxx     20    M        Chennai

And after sometimes it is updated as 

Name   Age    Sex   Address
YYY    25      F     Salem
ZZZ    18      M     Covai

I want this file to be written into hdfs using Apache flume whenever there is a change in the input file. Can any one suggest me the link or can any one tell me how to achieve this

Upvotes: 2

Views: 328

Answers (1)

Sravan K Reddy
Sravan K Reddy

Reputation: 1082

you can consider flume tail as a source.i am pointing some urls.please refere them.

   http://www.rittmanmead.com/2014/05/trickle-feeding-webserver-log-files-to-hdfs-using-apache-flume/

   http://www.ibm.com/developerworks/library/bd-flumews/

   http://mapredit.blogspot.in/2012/03/flumeng-evolution.html

    https://flume.apache.org/FlumeUserGuide.html

    http://stackoverflow.com/questions/13721930/flume-ng-tail-a-file

Upvotes: 2

Related Questions