CODEWITHSUNDEEP
apache-kafkabigdataapache-zookeeper
Eliane PDC
Eliane PDC

Reputation: 103

Start Zookeeper and Kafka Servers Files were unexpected

I'am beginner with kafka

1/ I Downloaded Kafka the 1.0.0 release

2/ I changed the data directory location property in bith server.properties and zookeeper.properties

\config\server.properties \config\zookeeper.properties

3/when I try to start Zookeeper and Kafka Servers I have an error " Files were unexpected "

D:\kafka-1.0.0-src\kafka-1.0.0-src\bin\windows>zookeeper-server-start.bat ....\config\zookeeper.properties Files était inattendu.

D:\kafka-1.0.0-src\kafka-1.0.0-src\bin\windows>kafka-server-start.bat ....\config\server.properties Files était inattendu.

Could you help me please?

Upvotes: 0

Views: 1331

Answers (3)

john k
john k

Reputation: 6614

I had to rewrite kafka-run-class.bat. The problem is spaces in file paths. You need to rewrite the command to handle spaces in variables. So put quotes around the JAVA variable, and you need quotes around everything in the classpath (the -cp flag).

all the call: concat %%i statements, I replaced that with a write into a new variable, kafkaCP, like so:

set kafkaCP=!kafkaCP!%%i;

Then I changed this line:

set COMMAND=%JAVA% %KAFKA_HEAP_OPTS% %KAFKA_JVM_PERFORMANCE_OPTS% %KAFKA_JMX_OPTS% %KAFKA_LOG4J_OPTS% -cp "%CLASSPATH% %KAFKA_OPTS% %kafkaCP%" %*

to add the new variable %kafkaCP%. This makes all the -cp variables to be in quotes, meaning it can handle spaces properly. Then the script ran correctly.

Upvotes: 0

Eliane PDC
Eliane PDC

Reputation: 103

Thank you Bhaskar for your reply, my problem was my en JAVA_HOME had some spaces in folder name "Program Files".

Upvotes: 0

Bhaskar Das
Bhaskar Das

Reputation: 682

I am assuming zookeeper & kafka in each user's own folder as 'kafkaflume'

There are 2 folders inside kafkaflume folder: one is zookeeper, another one is kafka Here a configuration file also given : flumekafka.conf

You need to edit this file as per your requirements.

First need to start zookeeper:

open a terminal, go zookeeper folder and start it :

bin/zkServer.sh start

open another terminal, go kafka folder and start it as :

bin/kafka-server-start.sh config/server.properties

open another terminal, go kafka folder and start producer program as :

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic  <topicname>

Now edit flumekafka.conf file as well as prepare hdfs folder for data loading.

Now run flume agent command from console.

======================= Configuration file for Flume:=======================

# Name the components on this agent
agent.sources = r1
agent.sinks =  k2
agent.channels = c1


# Describe/configure the source
agent.sources.r1.type = org.apache.flume.source.kafka.KafkaSource
agent.sources.r1.zookeeperConnect = localhost:2181
agent.sources.r1.topic = <topicname>
agent.sources.r1.groupId = group1
agent.sources.r1.channels = c1
agent.sources.r1.interceptors = i1
agent.sources.r1.interceptors.i1.type = timestamp
agent.sources.r1.kafka.consumer.timeout.ms = 10000



# Describing/Configuring the sink
 agent.sinks.k2.type = hdfs
 agent.sinks.k2.hdfs.path =  hdfs://localhost:8020/user/<username>/<foldername>/%y-%m-%d
 agent.sinks.k2.hdfs.rollInterval = 5
 agent.sinks.k2.hdfs.rollSize = 0
 agent.sinks.k2.hdfs.rollCount = 0
 agent.sinks.k2.hdfs.fileType = DataStream
 agent.sinks.k2.channel = c1


# Describing/Configuring the channel agent.channels.MemChannel.type = memory
 agent.channels.c1.type=memory
 agent.channels.c1.capacity = 10000
 agent.channels.c1.transactionCapacity = 1000

Upvotes: -1

Related Questions