CODEWITHSUNDEEP
apache-kafkakafka-producer-api
Javier Holguera
Javier Holguera

Reputation: 1351

Different between KafkaProducer.close() and KafkaProducer.flush()

Looking at the documentation, I'm not sure if I understand the difference between using close() and flush().

This is the doc for flush()

 * Invoking this method makes all buffered records immediately available to send (even if <code>linger.ms</code> is
 * greater than 0) and blocks on the completion of the requests associated with these records. The post-condition
 * of <code>flush()</code> is that any previously sent record will have completed (e.g. <code>Future.isDone() == true</code>).
 * A request is considered completed when it is successfully acknowledged
 * according to the <code>acks</code> configuration you have specified or else it results in an error.

And the doc for close():

 * This method waits up to <code>timeout</code> for the producer to complete the sending of all incomplete requests.
 * If the producer is unable to complete all requests before the timeout expires, this method will fail
 * any unsent and unacknowledged records immediately.

Does this mean that:

  1. If I use close() and there are some records pending in the in-memory buffer, they won't even be attempted (compared to flush, which would attempt to send them)?
  2. If I use flush(), it might block "forever" if the retries are large? While with close(), I have an upper bound for how long this is going to take?

I suppose if I'm right in 1., a producer with acks=0 would get a confirmation for a record that might not even be attempted to be published if it is "unlucky" enough to be placed in the in-memory queue and immediately after close() is called.

Upvotes: 6

Views: 5369

Answers (1)

varkashy
varkashy

Reputation: 385

If you want to keep using the producer and wait for messages to be sent, you would use flush else close. Close with timeout value will wait for the messages to be sent and ack received as per the config till time out value and then close the producer

Upvotes: 3

Related Questions