CODEWITHSUNDEEP
javaapache-kafkaspring-kafka
vijay
vijay

Reputation: 67

How to consume all kafka messages of one kafka topic every second from the beginning using spring boot?

I'm very new to kafka and I need to print all the messages from the kafka topic continuously(every second) from the start to end using spring boot.

Also need to get all the message at a time, not one by one.

I used.. auto.offset.reset=earliest and enable.auto.commit=false and random/new group.id but nothing works for me :(

@KafkaListener(topics = "retry-events", groupId = "my-consumer-group")
    public void consumeMessage(String message) {
        System.out.println("Received message: " + message);
    }

Here, If i send one message to kafka, thenits printing the latest message. I need to print every message again and again from the start each second.

Upvotes: 1

Views: 917

Answers (1)

Gary Russell
Gary Russell

Reputation: 174729

By default; Kafka maintains a last committed offset pointer for each consumer group/partition.

When the consumer starts, the default behavior is to start from that offset. auto.offset.reset=earliest only applies when there is no committed offset for a consumer (generally, the first time the consumer is started).

To always start from the beginning, you need to perform seek operations when the partitions are assigned.

When using Spring, this can easily be done by having the listener extend AbstractConsumerSeekAware and calling seekToBeginning() when the partitions are assigned.

See https://docs.spring.io/spring-kafka/docs/current/reference/html/#seek

Upvotes: 3

Related Questions