CODEWITHSUNDEEP
pythonapache-kafkakafka-python
Joy
Joy

Reputation: 4483

consuming message in client in kafka-python

I am new to Kafka. By taking help from a few online tutorials of kafka-python I have written following piece of code:

   from kafka import SimpleProducer, KafkaClient, KafkaConsumer
   kafka =  KafkaClient("localhost:9092")
   producer = SimpleProducer(kafka)
   producer.send_messages(b'my-topic', b'this method', b'Hello World')
   consumer = KafkaConsumer('my-topic',
                     group_id='my_group',
                     bootstrap_servers=['localhost:9092'])
   for message in consumer:
       print("%s:%d:%d: key=%s value=%s" % (message.topic, message.partition,
                                         message.offset, message.key,
                                         message.value))

But the problem is that in that last for loop code execution is stuck which I cannot figure out.

Upvotes: 2

Views: 5468

Answers (1)

BAE
BAE

Reputation: 8936

You original code is right. I run your code. KafkaConsumer can be used to consume messages. If you open another console, and run the same original code, you will see the output.

In http://kafka-python.readthedocs.org/en/latest/apidoc/kafka.consumer.html, there are lots of Consumer classes, such as SimpleConsumer, KafkaConsumer, Consumer, and so on. Why you code is stuck in the for-loop? Because the consumer in your code is set to consume new messages by default. In this case, the messages produced by producer.send_messages() function would not be consumed by the consumer.

By the way, if you use SimpleConsumer, you can use seek() to set the offset of messages you would like to consume.

Upvotes: 2

Related Questions