CODEWITHSUNDEEP
javascalaapache-kafkakafka-consumer-api
Count
Count

Reputation: 1423

Error creating Kafka topic :- replication factor larger than available brokers

I am trying to create a kafka topic via AdminCommand using below code Source

 ZkClient zkClient = new ZkClient(kafkaHost, 10000, 10000, ZKStringSerializer$.MODULE$);
    AdminUtils.createTopic(zkClient, "pa_reliancepoc_telecom_usageevent", 10, 2, new Properties());

But getting the below exception

Exception in thread "main" kafka.admin.AdminOperationException: replication factor: 1 larger than available brokers: 0
at kafka.admin.AdminUtils$.assignReplicasToBrokers(AdminUtils.scala:70)
at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:155)

However, I am able to create the topic using shell command .

Upvotes: 11

Views: 42006

Answers (3)

Ananth Joshi
Ananth Joshi

Reputation: 11

I faced the same issue while setting up multiple brokers.

The step I was missing was:

While editing the config file: config/server-1.properties: broker.id=1 listeners=PLAINTEXT://:9093 log.dir=/tmp/kafka-logs-1

need to update the LOG BASICS section as well(see below):

####################### Log Basics

A comma separated list of directories under which to store log files

log.dirs=/tmp/kafka-logs-1

Upvotes: 0

Jaya Ananthram
Jaya Ananthram

Reputation: 3463

In your code,

 AdminUtils.createTopic(zkClient, "pa_reliancepoc_telecom_usageevent", 10, 2, new Properties());

The fourth argument is the replication factor. So you are trying to create a topic with a name of pa_reliancepoc_telecom_usageevent with partition count of 10 and replication of 2. So two kafka brokers should be available while creating the topic. If less than two is available then you will get the following exception.

Exception in thread "main" kafka.admin.AdminOperationException: replication factor: 1 larger than available brokers: 0
at kafka.admin.AdminUtils$.assignReplicasToBrokers(AdminUtils.scala:70)
at kafka.admin.AdminUtils$.createTopic(AdminUtils.scala:155)

Make sure that you are running kafka cluster with two broker nodes and the two nodes should be alive while creating the topic.

To run kafka in cluster refer Step 6 in this link

Upvotes: 14

user2720864
user2720864

Reputation: 8161

configure your local machine to have multiple brokers up and running in case you decide to keep your replication_factor > 1.

You can do that by simply having multiple copies of the server.properties file. e.g
server-1.properties & server-2.properties

You need to then specify different broker.id & port in each of those files to make them unique ..

   config/server-1.properties:
     broker.id=1
     port=9093
     log.dir=/tmp/kafka-logs-1

   config/server-2.properties:
     broker.id=2
     port=9094
     log.dir=/tmp/kafka-logs-2

And then start multiple instances with following commands

> bin/kafka-server-start.sh config/server-1.properties &
> bin/kafka-server-start.sh config/server-2.properties &

For more details check Step 6: Setting up a multi-broker cluster

Upvotes: 5

Related Questions