CODEWITHSUNDEEP
javaapache-kafkakafka-consumer-api
Darshnik Swamy
Darshnik Swamy

Reputation: 71

How to throttle Kafka Consumer on the basis of QPS to be supported?

Use Case : Consume Data from Kafka, and then write to a DB which is bound by a max QPS.

Predicted solution : throttle the rate of consumption to a limit which is less the max write QPS of the DB.

Query: Guava gives a rate limiter which can be implemented here, but does Kafka supports consumer rate limiting itself rather than implementing third party options.

References for features on Kafka Rate Limiting : https://cwiki.apache.org/confluence/display/KAFKA/KIP-13+-+Quotas

Also, how can the quota rate limiting can be applied in this case ?

Upvotes: 3

Views: 10249

Answers (1)

Darshnik Swamy
Darshnik Swamy

Reputation: 71

The best solution is to use third-party libraries. Google's Guava is the best and the easiest one to use.

For sample codes, regarding the usage of Guava Rate Limiter, please refer this

Upvotes: 2

Related Questions