CODEWITHSUNDEEP
rabbitmqmasstransit
Alexey Koptyaev
Alexey Koptyaev

Reputation: 317

Keep messages in queue while consumer is offline

I use Masstransit in C# project.

I have a publisher and consumer services, and when both of them are up, then there are no problems. But if the consumer goes offline, published messages don't go to the queue. They just disappear.

The expected behavior is to keep messages in the queue until the consumer is started, and then send them to it. I've found several topics in google groups with same questions, but it wasn't clear for me how to solve that problem.

It seems strange to me that this functionality isn't provided out of the box because, in my understanding, it is the main purpose of RabbitMQ and MT.

The way I create publisher bus:

public static IBusControl CreateBus()
{
    return Bus.Factory.CreateUsingRabbitMq(sbc =>
    {
        var host = sbc.Host(new Uri("rabbitmq://RMQ-TEST"), h =>
        {
            h.Username("test");
            h.Password("test");
        });
        sbc.ReceiveEndpoint(host, "test_queue", ep =>
        {
            ep.Handler<IProductDescriptionChangedEvent>(
                content => content.CompleteTask);
        });
    });
}

And the consumer:

    public static void StartRmqBus()
    {
        var bus = Bus.Factory.CreateUsingRabbitMq(cfg =>
        {
            var host = cfg.Host(new Uri("rabbitmq://RMQ-TEST"), h =>
            {
                h.Username("test");
                h.Password("test");
            });

            cfg.ReceiveEndpoint(host, "test_queue", ep =>
            {
                ep.Consumer<ProductChangedConsumer>();
            });
        });

        bus.Start();
    }

EDIT:

Here is one more interesting feature: if I stop both services and manually put a message to the queue via admin interface of MT, the message is waiting in test_queue. But when I start publisher or consumer service, it falls to test_queue_error queue.

Upvotes: 1

Views: 3052

Answers (2)

Alexey Zimarev
Alexey Zimarev

Reputation: 19640

You use the same queue for published and consumer, plus publisher has a consumer for this message type, as you pointed out in your own answer.

If your publisher does not consume messages, it is better to remove the receiving endpoint from it at all and then your service will be send-only.

If you have several services, where each of them need to have their own consumers for the same message type - this is how pub-sub works and you must have different queues per service. This is described in the Common Gotchas section of the documentation. In such scenario, each service will get it's own copy of the published message.

If you have one queue - you get competing consumers and this scenario is only valid for horizontal scalability, where you run several instance of the same services to increase the number of processed messages if the processing is too slow. In such case all these instances will consume messages from the same queue. In this scenario only one instance will get a message.

Upvotes: 1

Alexey Koptyaev
Alexey Koptyaev

Reputation: 317

It seems like my publisher was set up incorrectly. After removing this part:

sbc.ReceiveEndpoint(host, "test_queue", ep =>
{
    ep.Handler<IProductDescriptionChangedEvent>(
        content => content.CompleteTask);
});

it started to work as expected. Looks like it consumed its own messages, that's why I didn't see messages in the queue when the consumer was down.

Upvotes: 0

Related Questions