CODEWITHSUNDEEP
hadoophdfs
user2775185
user2775185

Reputation: 1207

Best Practice or Tools to move file inside HDFS

we are doing POC on map-reduce to count the module usage of our app. Application is running on J2EE container located on separate box, hadoop installation is on separate box and we need to move the log files from app container box to hadoop box. below mentioned are some of the queries where help / advice is required.

  1. any tool / framework to move the log files from multiple location to HDFS ? what is best practice ?
  2. how to run this jobs of moving the files ? should it be be cron job or file size trigger to move to HDFS ?

Upvotes: 0

Views: 240

Answers (1)

Jasper
Jasper

Reputation: 8705

You need Apache Flume. Flume is a popular standard tool for moving log files into Hadoop HDFS. It could run as a cron job.

Upvotes: 2

Related Questions