CODEWITHSUNDEEP
apache-kafkakafka-python
Andrey Lifanov
Andrey Lifanov

Reputation: 91

No Brokers Available error when trying to connect to Kafka

I have a very strange problem when trying to connect locally to Kafka 0.10.0.0 using Python client on CentOS.

My connection options are pretty simple and default:

kafka_consumer = kafka.KafkaConsumer(
        bootstrap_servers=['localhost:9092'],
        client_id="python-test-consumer"
    )

When I manually set listeners option in Kafka's server.properties file like:

listeners=PLAINTEXT://localhost:9092

I get the kafka.errors.NoBrokersAvailable despite the fact that I can still easily connect to Kafka broker server with curl or other linux stuff.

No advertised.listeners or other deprecated advertised options help to solve the problem. Thus, the only state of configuration which is working is one without listeners. What is certainly unacceptable, because we need to setup local cluster somehow.

It seems that solution for this silly problem is simple and is wondering around, but we couldn't figure it ourselves.

Upvotes: 9

Views: 25952

Answers (2)

Keira McLaughlin
Keira McLaughlin

Reputation: 1

I was having a similar problem. Turns out the auto version discovery only works with versions of kafka 2.4 or older (see here: https://kafka-python.readthedocs.io/en/master/compatibility.html) and my broker was using a newer version.

By manually setting the kafka version, the problem was fixed for me:

kafka_consumer = kafka.KafkaConsumer(
    bootstrap_servers=['localhost:9092'],
    client_id="python-test-consumer",
    api_version="2.X.X"
)

Upvotes: 0

Maximiliano Guerra
Maximiliano Guerra

Reputation: 323

This may sound silly, but the exact same problem happened to me because of this:

I upgraded to Kafka 0.10.0.0 via brew (Mac package manager). Brew then suggests to run like this one-liner:

$ zookeeper-server-start /usr/local/etc/kafka/zookeeper.properties; kafka-server-start /usr/local/etc/kafka/server.properties

Instead of how I executed before:

$ zkServer start
$ kafka-server-start /usr/local/etc/kafka/server.properties

The approach suggested kept throwing those "No Brokers Available" errors in the client. Then I just split the command in two lines:

$ zookeeper-server-start /usr/local/etc/kafka/zookeeper.properties
$ kafka-server-start /usr/local/etc/kafka/server.properties

And everything works like before!

Sorry if this doesn't work for you, but I figured it was worth mentioning.

Upvotes: 4

Related Questions