CODEWITHSUNDEEP
pythonapache-kafkaconfluent-kafka-python
Kevin
Kevin

Reputation: 31

Issues with connecting to Kafka topic as a Consumer

I'm new to Kafka and am trying to get a simple consumer working for my application. I have tried getting python-kafka and confluent-kafka to work.

My consumer config properties are the following:

conf = {'bootstrap.servers': <bootstrap server>,
        'security.protocol': 'SASL_SSL',
        'sasl.mechanism': 'PLAIN',
        'group.id': 'mygroup',
        'sasl.username': <USERNAME>,
        'sasl.password': <PASSWORD>}

While I am able to connect to the server using the Kafka CLI, I am unable to get any messages using either python-kafka or confluent-kafka.

In confluent-kafka, I get a connection timed out error on repeat.

%4|1727131307.906|FAIL|rdkafka#consumer-1| [thrd:<bootstrap_server>/bootstrap]: <bootstrap_server>/bootstrap: Connection setup timed out in state CONNECT (after 30329ms in state CONNECT)

In python-kafka, I get one of two issues: either no response from the consumer at all (infinitely hangs), or a "No Brokers Available" error.

I was able to previously connect to this groups Kafka topic using the config properties above. However, their bootstrap servers changed and after changing the bootstrap servers I cannot connect. I'm being told the issue is on my end with my config, since I can connect using the CLI with my provided credentials.

I've tried changing my consumer config properties security protocol, setting the auto.offset.reset property, and connecting to the fallback bootstrap servers. Any help would be greatly appreciated!

Upvotes: 0

Views: 245

Answers (0)

Related Questions