CODEWITHSUNDEEP
apache-kafka
Afan
Afan

Reputation: 69

What is the command to list down all the available brokers in Apache Kafka?

I want to run a multi node cluster in Apache Kafka. I have made three server.properties files - server, server1 and server2. I have also given then different broker ids and different port numbers. Still on running the script kafka-topics.sh with a replication factor if 3 ,it throws an error saying that the replication factor:3 is larger than the number of brokers:0

I used this command :

bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 1 --topic my-replica-topic

Error shown is

Error while executing topic command replication factor: 3 larger than available brokers: 0 kafka.admin.AdminOperationException: replication factor: 3 larger than available brokers: 0 at kafka.admin.AdminUtils$.assignReplicasToBrokers(AdminUtils.scala:70) at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:171) at kafka.admin.TopicCommand$.createTopic(TopicCommand.scala:93) at kafka.admin.TopicCommand$.main(TopicCommand.scala:55) at kafka.admin.TopicCommand.main(TopicCommand.scala)

Could you let me know where I am going wrong ?

Upvotes: 3

Views: 12087

Answers (1)

Kevin Li
Kevin Li

Reputation: 2114

I think u should at least start 3 kafka server to make sure the number of brokers is larger or equal to the number of replication factor:

First we make a config file for each of the brokers:

cp config/server.properties config/server-1.properties

cp config/server.properties config/server-2.properties

Now edit these new files and set the following properties:

config/server-1.properties:
    broker.id=1
    listeners=PLAINTEXT://:9093
    log.dir=/tmp/kafka-logs-1


config/server-2.properties:
    broker.id=2
    listeners=PLAINTEXT://:9094
    log.dir=/tmp/kafka-logs-2

The broker.id property is the unique and permanent name of each node in the cluster. We have to override the port and log directory only because we are running these all on the same machine and we want to keep the brokers from all trying to register on the same port or overwrite each others data. We already have Zookeeper and our single node started, so we just need to start the two new nodes:

bin/kafka-server-start.sh config/server-1.properties

bin/kafka-server-start.sh config/server-2.properties

Now create a new topic with a replication factor of three:

bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 3 --partitions 1 --topic my-replicated-topic

Upvotes: 3

Related Questions