CODEWITHSUNDEEP
flumeflume-ng
user84592
user84592

Reputation: 4882

apache flume log directory

Just begin to learn Apache Flume. I follow the instructions on Flume official getting started website:

https://cwiki.apache.org/confluence/display/FLUME/Getting+Started

Almost everything is fine after follow the instructions on above link. But I could not find any log file afterwards. I suppose log file is under {flume.directory}/logs. Any idea to find flume log files?

Here comes my log4j.properties:

flume.root.logger=INFO,LOGFILE
flume.log.dir=./logs
flume.log.file=flume.log

log4j.logger.org.apache.flume.lifecycle = INFO
log4j.logger.org.jboss = WARN
log4j.logger.org.mortbay = INFO
log4j.logger.org.apache.avro.ipc.NettyTransceiver = WARN
log4j.logger.org.apache.hadoop = INFO
log4j.logger.org.apache.hadoop.hive = ERROR

# Define the root logger to the system property "flume.root.logger".
log4j.rootLogger=${flume.root.logger}


# Stock log4j rolling file appender
# Default log rotation configuration
log4j.appender.LOGFILE=org.apache.log4j.RollingFileAppender
log4j.appender.LOGFILE.MaxFileSize=100MB
log4j.appender.LOGFILE.MaxBackupIndex=10
log4j.appender.LOGFILE.File=${flume.log.dir}/${flume.log.file}
log4j.appender.LOGFILE.layout=org.apache.log4j.PatternLayout
log4j.appender.LOGFILE.layout.ConversionPattern=%d{dd MMM yyyy HH:mm:ss,SSS} %-5p [%t] (%C.%M:%L) %x - %m%n


# Warning: If you enable the following appender it will fill up your disk if you don't have a cleanup job!
# This uses the updated rolling file appender from log4j-extras that supports a reliable time-based rolling policy.
# See http://logging.apache.org/log4j/companions/extras/apidocs/org/apache/log4j/rolling/TimeBasedRollingPolicy.html
# Add "DAILY" to flume.root.logger above if you want to use this
log4j.appender.DAILY=org.apache.log4j.rolling.RollingFileAppender
log4j.appender.DAILY.rollingPolicy=org.apache.log4j.rolling.TimeBasedRollingPolicy
log4j.appender.DAILY.rollingPolicy.ActiveFileName=${flume.log.dir}/${flume.log.file}
log4j.appender.DAILY.rollingPolicy.FileNamePattern=${flume.log.dir}/${flume.log.file}.%d{yyyy-MM-dd}
log4j.appender.DAILY.layout=org.apache.log4j.PatternLayout
log4j.appender.DAILY.layout.ConversionPattern=%d{dd MMM yyyy HH:mm:ss,SSS} %-5p [%t] (%C.%M:%L) %x - %m%n


# console
# Add "console" to flume.root.logger above if you want to use this
log4j.appender.console=org.apache.log4j.ConsoleAppender
log4j.appender.console.target=System.err
log4j.appender.console.layout=org.apache.log4j.PatternLayout
log4j.appender.console.layout.ConversionPattern=%d (%t) [%p - %l] %m%n

Upvotes: 1

Views: 2558

Answers (2)

prayagupadhyay
prayagupadhyay

Reputation: 31252

Based on your log4j.properties, it would be in the same directory where you start your flume-agent, thats what ./ would do.

So, if you run from /home directory, it would be

/home/logs/flume.log

Also, you can use linux commands anytime to find that out,

find / -name flume.log ##search in whole disk

find . -name flume.log ##search in current directory

Upvotes: 1

Erms
Erms

Reputation: 365

Cause your configuration use a relative path :

flume.log.dir=./logs
log4j.appender.LOGFILE.File=${flume.log.dir}/${flume.log.file}

Flume use your current directory to log.
(the directory where you are when you launch flume)

You can use an absolute path for flume.log.dir if you want to force flume to log in that path, no matter where you launch it.

Upvotes: 1

Related Questions