Kombu+RabbitMQ: Check whether a queue is empty

Question

Architecture

Consider a system with DB records. Each record can be in a live or expired status; live records should be processed periodically using an external software module.

I have solved this using a classic producer - consumer architecture with Kombu and RabbitMQ. The producer fetches the records from the DB every few seconds, and the consumer handles them.

The problem

The number of live events greatly varies, and on peak hours the consumer can't handle the load and the queue is clogged with thousand of items.

I would like to make the system adaptive, so that the producer will not send new events to the consumer if the queue is empty.

What have I tried

• Searching the Kombu documentation / API
• Inspecting the Queue object
• Using the RabbitMQ REST API: http://<host>:<port/api/queues/<vhost>/<queue_name>. It works, but it's yet another mechanism to maintain, and I prefer an elegant solution within Kombu.

How do I check whether a RabbitMQ is empty using Python's Kombu?

Answer 1

You can call queue_declare() on the kombu Queue object.

According to the docs the function returns:

Returns a tuple containing 3 items:
        the name of the queue (essential for automatically-named queues)
        message count
        consumer count

Therefore you can do:

name, msg_count, consumer_count = queue.queue_declare()