CODEWITHSUNDEEP
javaapache-kafkaspring-kafka
goku736
goku736

Reputation: 412

Java set Kafka retention time in source code

I have the following problem. I need to set the retention time in Kafka for certain selected topics. I found a solution where I can set it with the following command: 

kafka-topics --zookeeper localhost:2181 --alter --topic topic-name --config retention.ms=-1

I checked in Kafka's Web UI and confirmed that it got changed.

If possible, I want to set the retention time in Java myself, but I can't seem to find the appropriate class/configuration to set the time. I thought I could get the information about the retention in the ProducerConfig class, but I couldn't find it there.

Is it even possible to set the retention time in Java and if possible, how can I get it done?

Thanks in advance!

Upvotes: 2

Views: 3151

Answers (2)

goku736
goku736

Reputation: 412

This works for me :)

    private void setRetentionTime(String topicName, int retentionTime) {
        ConfigResource resource = new ConfigResource(Type.TOPIC, topicName);

        Collection<ConfigEntry> entries = new ArrayList<>();
        entries.add(new ConfigEntry(TopicConfig.RETENTION_MS_CONFIG, String.valueOf(retentionTime)));

        Config config = new Config(entries);
        Map<ConfigResource, Config> configs = new HashMap<>();
        configs.put(resource, config);

        AdminClient client = kafkaConfig.createAdminClient();
        client.alterConfigs(configs);
    }

Upvotes: 8

Borislav Markov
Borislav Markov

Reputation: 1745

I know previously from Java you can do do a login as a client. It was not possible to change topic configuration.

Recently with the new versions of Kafka they introduced this and I think it is possible to do that.

https://kafka.apache.org/10/javadoc/org/apache/kafka/clients/admin/KafkaAdminClient.html#alterConfigs-java.util.Map-org.apache.kafka.clients.admin.AlterConfigsOptions-

I now the answer sounds a bit vague, but you have to be more specific if you want complete answer. I can extend the answer if you have exact version of Kafka you have, the libraries you use to connect to kafka, etc ...

I think it is not the best idea to change server side configuration from your application. This is more infrastructural thing, and should not be done runtime while you run the application.

Upvotes: 0

Related Questions