CODEWITHSUNDEEP
apache-kafka
Raj
Raj

Reputation: 1727

In apache kafka, what's the producer configuration to control the max wait period before getting response from kafka?

In our application, for each request ( a REST endpoint POST call) which is causing a data update/insert into the DB and we have to publish/produce a message to Kafka for downstream systems to consume and do the further processing.

And, the SLA for most of the POST requests in our application is 5 sec and most of our POST calls are synchronous. That means, any processing wrapped up inside of a POST call should be finished within 5 sec to avoid the timeout issues.

With this setup, If a take scenario where I've wrapped up a DB call and a call to produce a message onto Kafka in a POST call , my question is, how can I control/set the max time that my kafka producer can wait before it gets a response from the broker?

Upvotes: 1

Views: 1096

Answers (1)

H.Ç.T
H.Ç.T

Reputation: 3549

request.timeout.ms can be the config that you are looking for:

request.timeout.ms: The configuration controls the maximum amount of time the client will wait for the response of a request. If the response is not received before the timeout elapses the client will resend the request if necessary or fail the request if retries are exhausted. This should be larger than replica.lag.time.max.ms (a broker configuration) to reduce the possibility of message duplication due to unnecessary producer retries.

For version 2.1 or higher you should also consider to set delivery.timeout.ms.

delivery.timeout.ms: An upper bound on the time to report success or failure after a call to send() returns. This limits the total time that a record will be delayed prior to sending, the time to await acknowledgement from the broker (if expected), and the time allowed for retriable send failures. The producer may report failure to send a record earlier than this config if either an unrecoverable error is encountered, the retries have been exhausted, or the record is added to a batch which reached an earlier delivery expiration deadline. The value of this config should be greater than or equal to the sum of request.timeout.ms and linger.ms.

For more information about delivery.timeout.ms you can check this link.

Upvotes: 1

Related Questions