CODEWITHSUNDEEP
javaapache-kafkakafka-consumer-api
rookie
rookie

Reputation: 426

Kafka Consumer Assignment returns Empty Set

I have subscribed to a Kafka topic as below. I need to run some logic only after the consumer has been assigned a partition.

However consumer.assignment() comes back as an empty set no matter how long I wait. If I do not have the while loop and then do a consumer.poll() I do get the records from the topic.Can any one tell me why this is happening ?

    consumer.subscribe(topics);
    Set<TopicPartition> assigned=Collections.emptySet();
    while(isAssigned) {
          assigned = consumer.assignment();
          if(!assigned.isEmpty()) {
              isAssigned= false;
          }
    }

//consumer props
Properties props = new Properties();
        props.put("bootstrap.servers", "xxx:9092,yyy:9092");
        props.put("group.id", groupId);
        props.put("enable.auto.commit", "false");
        props.put("session.timeout.ms", "30000");
        props.put("key.deserializer", "org.apache.kafka.common.serialization.StringDeserializer");
        props.put("value.deserializer", "io.confluent.kafka.serializers.KafkaAvroDeserializer");
        props.put("schema.registry.url", "http://xxx:8081");
        props.put(KafkaAvroDeserializerConfig.SPECIFIC_AVRO_READER_CONFIG, true);
        props.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
        props.put("max.poll.records", "100");

Upvotes: 12

Views: 9832

Answers (1)

Mickael Maison
Mickael Maison

Reputation: 26885

Until you call poll(), the consumer is just idling.

Only after poll() is invoked, it will initiate a connection to the cluster, get assigned partitions and attempt to fetch messages.

This is mentioned in the Javadoc:

After subscribing to a set of topics, the consumer will automatically join the group when poll(Duration) is invoked.

Once you start calling poll(), you can use assignment() or even register a ConsumerRebalanceListener to be notify when partitions are assigned or revoked from your consumer instance.

Upvotes: 13

Related Questions