error while starting flume

Question

I am trying to use dynatrace as a source for flume and hadoop as sink,where dynatrace and flume is on same server and hadoop is on another server.

However,when starting flume,I am getting below error :

ERROR [conf-file-poller-0] (org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run:211) - Unhandled error java.lang.NoSuchMethodError: org.slf4j.spi.LocationAwareLogger.log(Lorg/slf4j/Marker;Ljava/lang/String;ILjava/lang/String;Ljava/lang/Throwable;)V at org.apache.commons.logging.impl.SLF4JLocationAwareLog.debug(SLF4JLocationAwareLog.java:120) at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.register(MetricsSystemImpl.java:220) at org.apache.hadoop.metrics2.MetricsSystem.register(MetricsSystem.java:54) at org.apache.hadoop.security.UserGroupInformation$UgiMetrics.create(UserGroupInformation.java:106) at org.apache.hadoop.security.UserGroupInformation.(UserGroupInformation.java:208) at org.apache.flume.sink.hdfs.HDFSEventSink.authenticate(HDFSEventSink.java:529) at org.apache.flume.sink.hdfs.HDFSEventSink.configure(HDFSEventSink.java:247) at org.apache.flume.conf.Configurables.configure(Configurables.java:41) at org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.loadSinks(PropertiesFileConfigurationProvider.java:373) at org.apache.flume.conf.properties.PropertiesFileConfigurationProvider.load(PropertiesFileConfigurationProvider.java:223) at org.apache.flume.conf.file.AbstractFileConfigurationProvider.doLoad(AbstractFileConfigurationProvider.java:123) at org.apache.flume.conf.file.AbstractFileConfigurationProvider.access$300(AbstractFileConfigurationProvider.java:38) at org.apache.flume.conf.file.AbstractFileConfigurationProvider$FileWatcherRunnable.run(AbstractFileConfigurationProvider.java:202) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745)

This is my flume configuration file :

# Name the components on this agent
QKDSK305.sources = HTTPSource
QKDSK305.sinks = PurePathSink UserActionSink VisitSink NullSink
QKDSK305.channels = PurePathChannel UserActionChannel VisitChannel NullChannel

# Describe/configure HTTPSource
QKDSK305.sources.HTTPSource.type = org.apache.flume.source.http.HTTPSource
QKDSK305.sources.HTTPSource.port = 4321
QKDSK305.sources.HTTPSource.handler = com.dynatrace.diagnostics.btexport.flume.BtExportHandler

# Describe sinks
QKDSK305.sinks.PurePathSink.type = hdfs
QKDSK305.sinks.PurePathSink.hdfs.path = hdfs://QKDSK303:9000/user/bts/pp
QKDSK305.sinks.PurePathSink.hdfs.fileType = DataStream
QKDSK305.sinks.PurePathSink.hdfs.filePrefix = export
QKDSK305.sinks.PurePathSink.hdfs.fileSuffix = .txt
QKDSK305.sinks.PurePathSink.hdfs.rollInterval = 120
QKDSK305.sinks.PurePathSink.hdfs.rollSize = 131072
QKDSK305.sinks.PurePathSink.serializer = com.dynatrace.diagnostics.btexport.flume.BtPurePathSerializerBuilder

QKDSK305.sinks.UserActionSink.type = hdfs
QKDSK305.sinks.UserActionSink.hdfs.path = hdfs://QKDSK303:9000/user/bts/pa
QKDSK305.sinks.UserActionSink.hdfs.fileType = DataStream
QKDSK305.sinks.UserActionSink.hdfs.filePrefix = export
QKDSK305.sinks.UserActionSink.hdfs.fileSuffix = .txt
QKDSK305.sinks.UserActionSink.hdfs.rollInterval = 120
QKDSK305.sinks.UserActionSink.hdfs.rollSize = 131072
QKDSK305.sinks.UserActionSink.serializer = com.dynatrace.diagnostics.btexport.flume.BtPageActionSerializerBuilder

QKDSK305.sinks.VisitSink.type = hdfs
QKDSK305.sinks.VisitSink.hdfs.path = hdfs://QKDSK303:9000/user/bts/visit
QKDSK305.sinks.VisitSink.hdfs.fileType = DataStream
QKDSK305.sinks.VisitSink.hdfs.filePrefix = export
QKDSK305.sinks.VisitSink.hdfs.fileSuffix = .txt
QKDSK305.sinks.VisitSink.hdfs.rollInterval = 120
QKDSK305.sinks.VisitSink.hdfs.rollSize = 131072
QKDSK305.sinks.VisitSink.serializer = com.dynatrace.diagnostics.btexport.flume.BtVisitSerializerBuilder
QKDSK305.sinks.VisitSink.serializer.charset = ISO-8859-2
QKDSK305.sinks.NullSink.type = null
QKDSK305.channels.NullChannel.type = memory

# Use a channel which buffers events in memory
QKDSK305.channels.PurePathChannel.type = memory
QKDSK305.channels.PurePathChannel.capacity = 1000
QKDSK305.channels.PurePathChannel.transactionCapactiy = 100

QKDSK305.channels.UserActionChannel.type = memory
QKDSK305.channels.UserActionChannel.capacity = 1000
QKDSK305.channels.UserActionChannel.transactionCapactiy = 100

QKDSK305.channels.VisitChannel.type = memory
QKDSK305.channels.VisitChannel.capacity = 1000
QKDSK305.channels.VisitChannel.transactionCapactiy = 100

# Bind the source and sink to the channel
QKDSK305.sources.HTTPSource.channels = PurePathChannel UserActionChannel VisitChannel NullChannel
QKDSK305.sinks.PurePathSink.channel = PurePathChannel
QKDSK305.sinks.UserActionSink.channel = UserActionChannel
QKDSK305.sinks.VisitSink.channel = VisitChannel
QKDSK305.sinks.NullSink.channel = NullChannel

QKDSK305.sources.HTTPSource.selector.type = multiplexing
QKDSK305.sources.HTTPSource.selector.header = btType
QKDSK305.sources.HTTPSource.selector.mapping.PUREPATH = PurePathChannel
QKDSK305.sources.HTTPSource.selector.mapping.PAGE_ACTION = UserActionChannel
QKDSK305.sources.HTTPSource.selector.mapping.VISIT = VisitChannel
QKDSK305.sources.HTTPSource.selector.default = NullChannel

Kindly help.

Thanks, Pranil

error while starting flume

Answers (1)

Related Questions