CODEWITHSUNDEEP
apache-kafkaapache-zookeepermessagebroker
Amol Dixit
Amol Dixit

Reputation: 669

Kafka Broker - Is it same as Zookeeper? or KafkaProducer is the broker?

There are multiple code examples available (One of such is here) where KafkaProducer and KafkaConsumers are implemented and can be started separately.

I was trying to find the code that starts the broker?

I think Zookeeper is not the KafkaBroker, as Zookeeper is for some other purpose .. managing clusters states.

While creating the KafkaProducer we pass in property like 

props.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, IKafkaConstants.KAFKA_BROKERS);

Does that mean the KafkaProducer it self start the broker?

Upvotes: 3

Views: 931

Answers (2)

Jhon Mario Lotero
Jhon Mario Lotero

Reputation: 325

Is important to begin have so clear the concepts:

Kafka broker = One instance of kafka started, i mean start the application download from (https://kafka.apache.org/downloads)

Zookeeper = Effectively is a coordinator for cluster in this specific case you can start up many instance on kafka with zookeeper and take a cluster.

Kafka Producer = Any application (implement kafka libraries) that is in charge of send message to specific kafka topic, some like queue.

Kafka consumer = Any application (implement kafka libraries) that is in charge of get messages of kafka cluster and process it.

For start kafka you first set a system environment (optional), and then execute this commands :

---- first start zookeeper

nohup zookeeper-server-start.sh $KAFKA_HOME/config/zookeeper.properties &

----- then istart kafka

nohup kafka-server-start.sh $KAFKA_HOME/config/server.properties &

Upvotes: 3

Simon Clark
Simon Clark

Reputation: 654

The Zookeeper manages the co-ordination and synchronisaton of the Kafka brokers, and they are separate components.

The Producer does not start the broker. The Producer and Consumer are client-side components that connect to one or more Kafka brokers, and produce/consume data to and from the cluster of brokers.

The Kafka broker is generally started by running kafka/bin/kafka-server-start.sh kafka/config/server.properties, which will start one Kafka broker with the properties defined in the server.properties file.

Upvotes: 5

Related Questions