CODEWITHSUNDEEP
javarabbitmq
Dojo
Dojo

Reputation: 5694

Rabbit MQ: Improve queue flushing speed

I have a durable queue which holds persistent messages. The messages arrive into the queue at a rate of about 10 messages per second.

The client is unable to fetch those messages at that rate. As a result the queue on the server keeps growing.

Each message is less than 1 KB and I have a healthy 2 Mbps line between the server and my machine. Using a network monitoring utility, I found that it is hardly using any of that bandwidth.

The client is doing nothing with the messages as of now, just printing them to console so processing time on client is almost 0.

Some other details: I am using a java client. I have set the client to prefetch 10000 messages. (also tried with default values) The round trip time is about 350 ms. Messages are individually acknowledged.

The available resources are being underutilized and 10 messages per second is hardly any load in my opinion. How do I speed up things so that messages held up in queue are transferred faster to the client. Possibly using some sort of batching.

Upvotes: 0

Views: 318

Answers (1)

Peter Lawrey
Peter Lawrey

Reputation: 533820

If you are indidivudally acknowledging messages every 350 ms, I would expect the consumer might achieve about 1/0.35 or about 2.9 messages per second. However, the protocol might not be that efficient and it may need two round trips to the server to acknowledge the message and get the next one. i.e. 1.4 message per second may be more realistic.

A round trip of 350 ms is very high, you can go around the world and back again in that time, so a simple solution may not work best for you. e.g. London -> New York -> Tokyo -> London.

I would try having a broker local to your client instead. This way the round trip is between your client and your local broker.

Upvotes: 1

Related Questions