CODEWITHSUNDEEP
apache-kafkafiware-orionfiware-cygnus
Charlotte
Charlotte

Reputation: 93

Apache kafka : broker leader -1 (topic received from Orion via Cygnus)

I'm working with Apache Kafka and receiving topics from Orion Context Broker via Cygnus (Fiware Labs)

I'm receiving 10 topics, and I can see data arriving in the consumer console for 8 topics.

But for 2 others topics, I cannot see any data arriving. And there is no error code (the consumer is just empty). If i try to add a test line to the topic via the producer console, i get this error:

ERROR Error when sending message to topic sensors_presence2_sensors with key: null, value: 4 bytes with error: Batch Expired (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)

So I used the describe command and I get this : 

Topic:sensors_presence2_sensors PartitionCount:1    ReplicationFactor:1 Configs:
Topic: sensors_presence2_sensors    Partition: 0    Leader: -1  Replicas: 2 Isr:

I'm just starting with Kafka, so for the moment I have 1 broker(0) and no partition. But why is my leader -1 ? This broker do not even exist. How can I change that ? I didn't choose the configuration for my topic, they arrived automatically from Cygnus (Orion Context Broker) with a OrionKafkaSink.

An example of one of the 8 topics that works well : 

Topic:sensors_presence1_sensors PartitionCount:1    ReplicationFactor:1 Configs:
Topic:sensors_presence1_sensors Partition: 0    Leader: 0   Replicas: 0 Isr: 0

Thanks

Edit : In Cygnus logs, it shows that data is correctly send to kafka : 

time=2016-03-02T11:07:09.504UTC | lvl=INFO | trans=1456915468-194-0000000039 | srv=egmmqtt | subsrv=egmmqttpath | function=persistAggregation | comp=Cygnus | msg=com.telefonica.iot.cygnus.sinks.OrionKafkaSink[279] : [kafka-sink] Persisting data at OrionKafkaSink. Topic (sensors_presence2_sensors), Data (...

Upvotes: 2

Views: 1905

Answers (1)

avr
avr

Reputation: 4893

The result of describe command is showing Replicas:2 and Isr: (empty) that means broker with id 2 was active at the time of creation of that topic and the same broker(id=2) is not active now. Because of that Isr( In sync replicas) showing empty.

There is no chance of getting Replicas: 2 when you have only one node (broker-id=0) kafka cluster. Make broker-2 up and everything will works well.

Hope this helps!

Upvotes: 0

Related Questions