CODEWITHSUNDEEP
rabbitmq
D. Ziegler
D. Ziegler

Reputation: 23

RabbitMQ more messages than expected on fixed size queue

I have a publisher that sends messages to a consumer that moves a motor. The motor has a work queue which I cannot access, and it works slower than the rate of the incoming messages, so I'm trying to control the traffic on the consumer.

To keep updated and relevant data coming to the motor without the queue filling up and creating a traffic jam, I set the RabbitMQ queue size limit to 5 and basicQos to 1.

The idea is that the RabbitMQ queue will drop the old messages when it is filled up, so the newest commands are at the front of the queue. Also by setting basicQos to 1 I ensure that the consumer doesn't grab all messages from the queue and bombards the motor at once, which is exactly what i'm trying to avoid since I can't do anything once the command was sent to the motor. This way the consumer takes messages from the queue one by one, while new messages replace the old ones on the queue. Practically this moves the bottleneck to the RabbitMQ queue instead of the motor's queue. I also cannot check the motor's work queue, so all traffic control must be done on the consumer.

I added messageId and tested, and found out many messages are still coming and going long after the publisher is being shut down.

I'm expecting around 5 messages after shutdown since that's the size of the queue, but i'm getting hundreds.

I also added a few seconds of sleep inside the callback to make sure this isn't the robot queue that's acting up, but i'm still getting many messages after shutdown, and I can see in the logs the callback is being called every time so it's definitely still getting messages from somewhere.

Please help. Thanks.

Upvotes: 1

Views: 592

Answers (1)

D. Ziegler
D. Ziegler

Reputation: 23

Moving the acknowledgment to the end of the callback solved the problem.

I'm guessing that by setting basicQos to 1 it did execute the callback for each message one after another, but in the background it kept grabbing messages from the queue. So even when the publisher was shutdown, the consumer still had messages that were taken from the queue in it, and those messages were the ones that I saw being executed.

Upvotes: 1

Related Questions