CODEWITHSUNDEEP
apache-kafkaconsumer
Arnon Rodman
Arnon Rodman

Reputation: 541

Apache Kafka consumer by date

Is there a low-level or high-level consumer, Kafka 0.8.2, that can consume from a specific time period /date? ( Kafka-client, spark-Kafka,...)

Upvotes: 1

Views: 1153

Answers (2)

Raghava Reddy
Raghava Reddy

Reputation: 343

Use OffsetAndTimestamp:

ConsumerRecords<String, String> records = consumer.poll(Duration.ofMillis(100));
            Set<TopicPartition> assignments = consumer.assignment();
            Map<TopicPartition, Long> query = new HashMap<>();
            for (TopicPartition topicPartition : assignments) {
                query.put(topicPartition, date);
            }

            Map<TopicPartition, OffsetAndTimestamp> result = consumer.offsetsForTimes(query);

            result.entrySet().stream().forEach(entry -> consumer.seek(entry.getKey(),
                    Optional.ofNullable(entry.getValue()).map(OffsetAndTimestamp::offset).orElse(new Long(0))));

            flag = false;

        for (ConsumerRecord<String, String> record : records)
            System.out.printf("offset = %d, key = %s, value = %s%n", record.offset(), record.key(), record.value());

Upvotes: 1

dawsaw
dawsaw

Reputation: 2313

No there is not. You must go to 0.10 in order to get this kind of functionality.

Upvotes: 0

Related Questions