Dhinesh
Reputation: 325
How to get the latest message offset from the FlinkKafkaConsumer?
I am using FlinkKafkaConsumer to consume the data from kafka? but I am not able to get the latest offset from he data received. How to do it in flink?.Could anyone guide me in this?
Upvotes: 1
Views: 635
Answers (1)
Robert Metzger
Reputation: 4542
You have to use a custom deserialziation schema called KeyedDeserializationSchema to access the offset for each message.
The deserialize() method has an argument where the framework passes you the offset for each message.
Upvotes: 3
Related Questions
- How to get latest offset/size of a Kafka topic using KafkaAdminClient (Java) for 2.x version
- How do I get the the offset of last message of a Kafka topic using confluent-kafka-python?
- How to seek to the last offset in Kafka correctly?
- How to consume only latest offset in Kafka topic
- How to consume kafka message from server with offset value?
- How to read first N kafka messages in flink?
- How to set Kafka offset for consumer?
- Can I retrieve the latest available offset for a Kafka partition without retrieving all the messages?
- read kafka message starting from a specific offset using high level API
- Kafka - Simplest Way to Get Latest Offset