Reputation: 10876
How can I reduce the 1s consumer lag in the Kafka Tutorial?
I'm working through the very first section of the Confluent Tutorials: https://kafka-tutorials.confluent.io/kafka-console-consumer-producer-basics/kafka.html. Everything works as described, but I notice there's about 1 second of lag between when I press enter in the producer terminal and when a message is displayed in the consumer terminal. Is it the producer or the consumer who's responsible for this lag/batching? Is there a way to configure things to be more responsive? A quick search turned up the linger.ms setting, but it seems like recent versions of Kafka default this setting to zero, and it doesn't appear to be overridden in these containers.
Upvotes: 0
Views: 80
Answers (1)
Reputation: 10876
Ok, it looks like setting --timeout=0 in the producer makes the lag disappear. Looking at the kafka-console-producer source code, --timeout defaults to 1000 and gets merged into LINGER_MS_CONFIG. So even though linger defaults to zero in Kafka generally, it effectively defaults to 1 sec in this command line producer.
Upvotes: 1
Related Questions
- Advice on how I can decrease Kafka Lag
- how to fix kafka consumer lag in 0.10.2.1 kafka
- kafka consumer lag increases too much
- Delay in Consumer consuming messages in Apache Kafka
- How to reduce latency between kafka's producer and consumer?
- Kafka Reduce Lag for Consumer
- How can i reduce lag in kafka consumer/producer
- Kafka consumers reading too slow
- Kafka latency optimization
- Kafka is slow to produce messages in first seconds