CODEWITHSUNDEEP
apache-kafkaapache-kafka-connect
Omri. B
Omri. B

Reputation: 462

Kafka connect - completely removing a connector

my question is split to two. I've read Kafka Connect - Delete Connector with configs?. I'd like to completely remove a connector, with offsets and all, so I can recreate it with the same name later. Is this possible? To my understanding, a tombstone message will kill this connector indefinitely.

The second part is - is there a way to have the kafka-connect container automatically delete all connectors he created when bringing it down? Thanks

Upvotes: 2

Views: 2104

Answers (1)

OneCricketeer
OneCricketeer

Reputation: 191874

There is no such command to completely cleanup connector state. For sink connectors, you can use kafka-consumer-groups to reset it's offsets. For source connectors, it's not as straightforward, as you'll need to manually produce data into the Connect-managed offsets topic.

The config and status topics also persist historical data, but shouldn't prevent you from recreating the connector with the same name/details.

The Connect containers published by Confluent and Debezium always uses Distributed mode. You'll need to override the entrypoint of the container to use standalone mode to not persist the connector metadata in Kafka topics (this won't be fault tolerant, but it'll be fine for testing)

Upvotes: 2

Related Questions