CODEWITHSUNDEEP
amqp
David Wolever
David Wolever

Reputation: 154644

Is it possible to declare a maximum queue size with AMQP?

As the title says — is it possible to declare a maximum queue size and broker behaviour when this maximum size is reached? Or is this a broker-specific option?

I ask because I'm trying to learn about AMQP, not because I have this specific problem with any specific broker… But broker-specific answers would still be insightful.

Upvotes: 6

Views: 6596

Answers (5)

Alexander Bartosh
Alexander Bartosh

Reputation: 8857

You can read about RabbitMQ maximum queue implementation here http://www.rabbitmq.com/maxlength.html

They do not block the incoming messages addition but drop the messages from the head of the queue.

You should definitely read about Flow control here: http://www.rabbitmq.com/memory.html

Upvotes: 2

Michael Dillon
Michael Dillon

Reputation: 32392

There are some things that you can't do with brokers, but you can do in your app. For instance, there are two AMQP methods, basic.get and queue.declare, which return the number of messages in the queue. You can use this to periodically get a count of outstanding messages and take action (like start new consumer processes) if the message count gets too high.

Upvotes: 0

Sam Stokes
Sam Stokes

Reputation: 14817

Depending on why you're asking, you might not actually need a maximum queue size. Since version 2.0 RabbitMQ will seamlessly persist large queues to disk instead of storing all the messages in RAM. So if your concern the broker crashing because it exhausts its resources, this actually isn't much of a problem in most circumstances - assuming you aren't strapped for hard disk space.

In general this persistence actually has very little performance impact, because by definition the only "hot" parts of the queue are the head and tail, which stay in RAM; the majority of the backlog is "cold" so it makes little difference that it's sitting on disk instead.

We've recently discovered that at high throughput it isn't quite that simple - under some circumstances the throughput can deteriorate as the queue grows, which can lead to unbounded queue growth. But when that happens is a function of CPU, and we went for quite some time without hitting it.

Upvotes: 6

jale
jale

Reputation: 31

With qpid, yes you can confire maximun queue size and politic in case raise the maximum. Ring, ignore messages,broke connection. you also have lvq queues (las value) very configurable

Upvotes: 1

old_sound
old_sound

Reputation: 2313

AFAIK you can't declare maximum queue size with RabbitMQ.

Also there's no such setting in the AMQP sepc:

http://www.rabbitmq.com/amqp-0-9-1-quickref.html#queue.declare

Upvotes: 6

Related Questions