Reputation: 5070
What I would like to achieve:
I would like to identify the retried message using Spring Kafka when producing the retried message.
Background:
I am a Spring Kafka producer.
I produce messages that I put in a kafka broker.
Unfortunately, the kafka cluster is flaky, but not to a point where it is completely down. Usually, couple of retries will send the message successfully to Kafka.
The different consumers of the topic have the requirement to identify a message that has been retried.
Meaning, when they get a message I managed to produce without problems, no need for anything else.
However, if the message has been produced, but went thought the retry mechanism, I would like to identify / tag those a such.
Here is what I tried:
retries (defaults to Integer.MAX_VALUE): the maximum number of attempts to publish the message
delivery.timeout.ms (defaults to 120,000): the maximum time to wait for a message to be acknowledged before considering it failed
retry.backoff.ms (defaults to 100): the time to wait before retrying
retry.backoff.max.ms (defaults to 1,000): the maximum delay between consecutive retries
With this from the official doc, I went to use this construct:
Properties props = new Properties();
// other properties
props.put(RETRIES_CONFIG, 20);
props.put(RETRY_BACKOFF_MS_CONFIG, "500");
props.put(DELIVERY_TIMEOUT_MS_CONFIG, "5000");
[...]
props.put(BOOTSTRAP_SERVERS_CONFIG, KAFKA_CONTAINER.getBootstrapServers());
props.put(KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
props.put(VALUE_SERIALIZER_CLASS_CONFIG, StringSerializer.class.getName());
KafkaProducer<String, String> producer = new KafkaProducer<>(props);
While this would configure the retry, it does not distinguish a retried message.
Question:
How to distinguish, identify, tag that a message has been retried?
Upvotes: 0
Views: 33