CODEWITHSUNDEEP
apache-kafkacloudera
Prashant
Prashant

Reputation: 772

kafka console consumer not receiving messages from console producer

I recently setup a cloudera quickstartVM using docker image and setup Kafka parcel in it. After successful installation, i see that all the services are running in green status (including Kafka and zookeeper). However, when I follow the below commands of kafka CLI i don't see consumer getting messages.

Any help is greatly appreciated CDHv 5.13 CDK 4.0 (kafka 2.1)- through parcel.

kafka-topics --create quickstart.cloudera:9092 --replication-factor 1 --partitions 1 --topic test3 --zookeeper quickstart.cloudera:2181

topic created successfully on console.

Console Consumer (CLI terminal 1):
kafka-console-consumer --bootstrap-server quickstart.cloudera:9092 --topic test3

consumer started on console in Terminal 1

Console Producer:
kafka-console-producer --broker-list  quickstart.cloudera:9092 --topic test3

Producer created in Terminal 2. Now, when i type anything in console of Terminal 2 (producer), the consumer terminal doesn't show anything.

Please suggest what is missing here. I am not sure how to debug this situation.

I don't see any exception in the /var/log/kafka/kafka-broker-quickstart.cloudera.log file

19/05/07 09:18:28 INFO zookeeper.ClientCnxn: Socket connection established, initiating session, client: /172.17.0.2:60968, server: quickstart.cloudera/172.17.0.2:2181
19/05/07 09:18:28 INFO zookeeper.ClientCnxn: Session establishment complete on server quickstart.cloudera/172.17.0.2:2181, sessionid = 0x16a915bca140112, negotiated timeout = 30000
19/05/07 09:18:28 INFO zookeeper.ZooKeeperClient: [ZooKeeperClient] Connected.
Topic:test4     PartitionCount:1        ReplicationFactor:1     Configs:
        Topic: test4    Partition: 0    Leader: 37      Replicas: 37    Isr: 37
19/05/07 09:18:28 INFO zookeeper.ZooKeeperClient: [ZooKeeperClient] Closing.
19/05/07 09:18:28 INFO zookeeper.ClientCnxn: EventThread shut down
19/05/07 09:18:28 INFO zookeeper.ZooKeeper: Session: 0x16a915bca140112 closed
19/05/07 09:18:28 INFO zookeeper.ZooKeeperClient: [ZooKeeperClient] Closed.

This is the output of 

kafka-topics --zookeeper quickstart.cloudera:2181 --describe --topic test3

Upvotes: 0

Views: 774

Answers (2)

Ravi G
Ravi G

Reputation: 11

IFF using cloudera quickstart VM, in cloudera manager -> kafka configuration, change Offset Commit Topic Replication Factor (offsets.topic.replication.factor) to 1

Upvotes: 1

Prashant
Prashant

Reputation: 772

I removed the CDK4.0 parcel and instead added CDK 3.1 parcel. Now, the console consumer is working. Not sure what is the issue with CDK 4.0. CDK 3.1 brings in Kafka 1.0.1

Upvotes: 0

Related Questions