CODEWITHSUNDEEP
apache-kafkaapache-kafka-connect
Brady Maf
Brady Maf

Reputation: 1789

how to consume message from specific offset using Kafka HTTP Sink Connector

I'm using Kafka Confluent HTTP Sink Connector to consume the message from Kafka topic, by default it read all the messages from beginning. I've use case where I want to stop the connector for some time and resume the HTTP sink connector later. When I resume the connector I want to read the message from specific offset.

I was looking offset related configuration here https://docs.confluent.io/kafka-connect-http/current/connector_config.html but I couldn't find any.

How can I consume the message from specific offset?

Upvotes: 2

Views: 955

Answers (1)

OneCricketeer
OneCricketeer

Reputation: 192023

All (at least most) sink connectors use Kafka Consumer groups.

You can use kafka-consumer-groups --reset-offsets command to first seek the group for the connector to specific offsets. And then, when it starts again, it should begin to read from that position.

by default it read all the messages from beginning

Not necessarily. By default, auto.offset.reset=latest

Upvotes: 0

Related Questions