Reputation: 33
The application that I work for has some moderate amounts of 15k-17k req/sec events to be published to event-hub in Azure,
The event configuration looks as below:
metadata.max.age.ms: 180000
connections.max.idle.ms: 180000
max.request.size: 1000000
retries: 0
linger.ms: 150
delivery.timeout.ms: 60150
heartbeat.interval.ms: 3000
session.timeout.ms: 35000
batch.size: 163840 #160*1024 - 160kb is the batch size.
request.timeout.ms: 60000
sync: true
I use spring-messaging client to send events, it is synchronous and takes its time to receive an acknowledgement.
The main problem I face is when an event is created and pushed to Azure event hub, the Kafka takes more than 25 secs some times to send an acknowledgement though the event already reached Azure Event-Hubs within 2 to 3 secs, the checked the traces in the streaming app to verify that fact. Events are waiting for 30s (allowed timeout).
I'm stuck here, since I feel the configuration looks good from my point, why is the acknowledgement received with a delay? There is no problem with the event hub in Azure, it looks perfectly normal.
Upvotes: 1
Views: 268