CODEWITHSUNDEEP
spring-cloudspring-cloud-stream
Karthik Palanivelu
Karthik Palanivelu

Reputation: 118

SpringCloudStream - Slow Consumer for RabbitMQ binder

I have a usecase to send http POST to a HTTP Source created as Spring Boot APP with http Source of Spring Cloud Stream App Starter. This process in publishing 5k records/sec. I have a Sink Application to persist the data to Mongo DB. Read in the app is very slow of 20 msgs/sec. I am using the following properties and see no difference. I am using the same prefix to load properties - spring.cloud.stream.rabbit.binder. Can you please let me know how to achieve concurrency in the read of data from RabbitMQ binder?

application.properties

spring.cloud.stream.binder.rabbit.default.prefix=z.
spring.cloud.stream.bindings.input.destination=http-source
spring.cloud.stream.bindings.input.durableSubscription=true
spring.cloud.stream.bindings.input.group=default
spring.cloud.stream.rabbit.binder.addresses=localhost:5672
spring.cloud.stream.rabbit.binder.username=guest
spring.cloud.stream.rabbit.binder.password=guest
spring.cloud.stream.rabbit.binder.listener.concurrency=100
spring.cloud.stream.rabbit.binder.listener.max-concurrency=500
spring.cloud.stream.rabbit.binder.listener.prefetch=1000
spring.cloud.stream.rabbit.binder.listener.acknowledge-mode=NONE

server.port=${listen.port}

####################################################
# Mongo
#      Configuration - DEV
####################################################
mongodbDatabasename=*****
mongodbPassword=*****
mongodbUsername=*****
mongodbReplicaName=
mongodbAddresses=localhost:27017
mongodbAuthenticationDatabase=users
mongodbAuthMechanism=SCRAM-SHA-1
region=DEV
collectionName=*****
mongodbSocketTimeout=25000
mongodbConnectionTimeout=5000
maxConnectionForHost=5
minConnectionForHost=100

Thanks and Appreciate your help Karthik

Upvotes: 1

Views: 752

Answers (1)

Ilayaperumal Gopinathan
Ilayaperumal Gopinathan

Reputation: 4179

I believe you need to set the concurrency and other consumer related properties as per-binding consumer properties (with the prefix: spring.cloud.stream.rabbit.bindings.<channelName>.consumer.. You can refer more detail here

Not sure how did you come up with the properties with the prefix spring.cloud.stream.rabbit.binder.listener.concurrency. Do you see this anywhere in the documentation?

Upvotes: 1

Related Questions