CODEWITHSUNDEEP
hadoopflumeflume-ngflume-twitterbigdata
srk
srk

Reputation: 639

Any idea about how to install flume with hadoop in windows?

I install hadoop in c:\hadoop2.5.2 and then unzip flume binary file in it under c:\hadop2.5.2\apache-flume-1.6.0

I am getting the error as shown below any idea ?

C:\hadoop2.5.2\apache-flume-1.6.0\bin>flume-ng agent -conf -f conf/flume-twitter.conf -Dflume.root.logger=DEBUG,console -n TwitterAgent

C:\hadoop2.5.2\apache-flume-1.6.0\bin>powershell.exe -NoProfile -InputFormat none -ExecutionPolicy unrestricted -File C:\hadoop2.5.2\apache-
flume-1.6.0\bin\flume-ng.ps1 agent -conf -f conf/flume-twitter.conf -Dflume.root.logger=DEBUG,console -n TwitterAgent
C:\hadoop2.5.2\apache-flume-1.6.0\bin\flume-ng.ps1 : A parameter cannot be found that matches parameter name 'Dflume.root.logger=DEBUG,cons
ole'.
At line:0 char:1
+  <<<<
    + CategoryInfo          : InvalidArgument: (:) [flume-ng.ps1], ParentContainsErrorRecordException
    + FullyQualifiedErrorId : NamedParameterNotFound,flume-ng.ps1

C:\hadoop2.5.2\apache-flume-1.6.0\bin>

I have set parameters in flume.sh 

export JAVA_HOME=C:\Java\jdk1.7.0_79\

# Give Flume more memory and pre-allocate, enable remote monitoring via JMX
export JAVA_OPTS="-Xms100m -Xmx2000m -Dcom.sun.management.jmxremote"

# Note that the Flume conf directory is always included in the classpath.
FLUME_CLASSPATH="C:\hadoop2.5.2\apache-flume-1.6.0\lib\flume-sources-1.0-SNAPSHOT.jar"

Please guys any idea how to solve this ?

Upvotes: 0

Views: 984

Answers (1)

NangSaigon
NangSaigon

Reputation: 1263

I've gotten the same issue. It seems the document on the Flume website are outdated. Here is the example I have tried and it works for me.

bin\flume-ng agent -conf conf -conf-file conf/flume-conf.properties --name agent1 -property "flume.root.logger=INFO,console"

Replace -D by -property Let reference source code of Apache Flume on the Github. See from the line 20.

So, you should try:

bin\flume-ng agent -conf -f conf/flume-twitter.conf -property "flume.root.logger=INFO,console" -n TwitterAgent

Upvotes: 2

Related Questions