Servicestack RabbitMQ: Infinite loop fills up dead-letter-queue when RabbitMqProducer cannot redeclare temporary queue in RPC-pattern

Question

When I declare a temporary reply queue to be exclusive (e.g. anonymous queue (exclusive=true, autodelete=true) in rpc-pattern), the response message cannot be posted to the specified reply queue (e.g. message.replyTo="amq.gen-Jg_tv8QYxtEQhq0tF30vAA") because RabbitMqProducer.PublishMessage() tries to redeclare the queue with different parameters (exclusive=false), which understandably results in an error.

Unfortunately, the erroneous call to channel.RegisterQueue(queueName) in RabbitMqProducer.PublishMessage() seems to nack the request message in the incoming queue so that, when ServiceStack.Messaging.MessageHandler.DefaultInExceptionHandler tries to acknowlege the request message (to remove it from the incoming queue), the message just stays on top of the incoming queue and gets processed all over again. This procedure repeats indefinitely and results in one dlq-message per iteration which slowly fills up the dlq.

I am wondering,

• if ServiceStack handles the case, when ServiceStack.RabbitMq.RabbitMqProducer cannot declare the response queue, correctly
• if ServiceStack.RabbitMq.RabbitMqProducer muss always declare the response queue before publishing the response
• if it wouldn't be best to have some configuration flag to omit all exchange and queue declaration calls (outside of the first initialization). The RabbitMqProducer would just assume every queue/exchange to be properly set up and just publish the message.

(At the moment our client just declares its response queue to be exclusive=false and everything works fine. But I'd really like to use rabbitmq's built-in temporary queues.)

MQ-Client Code, requires simple "SayHello" service:

        const string INQ_QUEUE_NAME = "mq:SayHello.inq";
        const string EXCHANGE_NAME="mx.servicestack";

        var factory = new ConnectionFactory() { HostName = "192.168.179.110" };
        using (var connection = factory.CreateConnection())
        {
            using (var channel = connection.CreateModel())
            {
                // Create temporary queue and setup bindings

                // this works (because "mq:tmp:" stops RabbitMqProducer from redeclaring response queue)
                string responseQueueName = "mq:tmp:SayHello_" + Guid.NewGuid().ToString() + ".inq";
                channel.QueueDeclare(responseQueueName, false, false, true, null);

                // this does NOT work (RabbitMqProducer tries to declare queue again => error):
                //string responseQueueName = Guid.NewGuid().ToString() + ".inq";
                //channel.QueueDeclare(responseQueueName, false, false, true, null);

                // this does NOT work either (RabbitMqProducer tries to declare queue again => error)
                //var responseQueueName = channel.QueueDeclare().QueueName;

                // publish simple SayHello-Request to standard servicestack exchange ("mx.servicestack") with routing key "mq:SayHello.inq":
                var props = channel.CreateBasicProperties();
                props.ReplyTo = responseQueueName;
                channel.BasicPublish(EXCHANGE_NAME, INQ_QUEUE_NAME, props, Encoding.UTF8.GetBytes("{\"ToName\": \"Chris\"}"));

                // consume response from response queue
                var consumer = new QueueingBasicConsumer(channel);
                channel.BasicConsume(responseQueueName, true, consumer);
                var ea = (BasicDeliverEventArgs)consumer.Queue.Dequeue();

                // print result: should be "Hello, Chris!"
                Console.WriteLine(Encoding.UTF8.GetString(ea.Body));
            }
        }

Everything seems to work fine when RabbitMqProducer does not try to declare the queues, like that:

    public void PublishMessage(string exchange, string routingKey, IBasicProperties basicProperties, byte[] body)
    {
        const bool MustDeclareQueue = false; // new config parameter??
        try
        {
            if (MustDeclareQueue && !Queues.Contains(routingKey))
            {
                Channel.RegisterQueueByName(routingKey);
                Queues = new HashSet<string>(Queues) { routingKey };
            }

            Channel.BasicPublish(exchange, routingKey, basicProperties, body);
        }
        catch (OperationInterruptedException ex)
        {
            if (ex.Is404())
            {
                Channel.RegisterExchangeByName(exchange);

                Channel.BasicPublish(exchange, routingKey, basicProperties, body);
            }
            throw;
        }
    }

Answer 1

The issue got adressed in servicestack's version v4.0.32 (fixed in this commit).

The RabbitMqProducer no longer tries to redeclare temporary queues and instead assumes that the reply queue already exist (which solves my problem.)

(The underlying cause of the infinite loop (wrong error handling while publishing response message) probably still exists.)

Edit: Example

The following basic mq-client (which does not use ServiceStackmq client and instead depends directly on rabbitmq's .net-library; it uses ServiceStack.Text for serialization though) can perform generic RPCs:

    public class MqClient : IDisposable
    {
        ConnectionFactory factory = new ConnectionFactory()
        {
            HostName = "192.168.97.201",
            UserName = "guest",
            Password = "guest",
            //VirtualHost = "test",
            Port = AmqpTcpEndpoint.UseDefaultPort,
        };

        private IConnection connection;
        private string exchangeName;

        public MqClient(string defaultExchange)
        {
            this.exchangeName = defaultExchange;
            this.connection = factory.CreateConnection();
        }

        public TResponse RpcCall<TResponse>(IReturn<TResponse> reqDto, string exchange = null)
        {
            using (var channel = connection.CreateModel())
            {
                string inq_queue_name = string.Format("mq:{0}.inq", reqDto.GetType().Name);

                string responseQueueName = channel.QueueDeclare().QueueName;

                var props = channel.CreateBasicProperties();
                props.ReplyTo = responseQueueName;

                var message = ServiceStack.Text.JsonSerializer.SerializeToString(reqDto);

                channel.BasicPublish(exchange ?? this.exchangeName, inq_queue_name, props, UTF8Encoding.UTF8.GetBytes(message));

                var consumer = new QueueingBasicConsumer(channel);
                channel.BasicConsume(responseQueueName, true, consumer);


                var ea = (BasicDeliverEventArgs)consumer.Queue.Dequeue();
                //channel.BasicAck(ea.DeliveryTag, false);

                string response = UTF8Encoding.UTF8.GetString(ea.Body);
                string responseType = ea.BasicProperties.Type;
                Console.WriteLine(" [x] New Message of Type '{1}' Received:{2}{0}", response, responseType, Environment.NewLine);

                return ServiceStack.Text.JsonSerializer.DeserializeFromString<TResponse>(response);

            }
        }

        ~MqClient()
        {
            this.Dispose();
        }

        public void Dispose()
        {
            if (connection != null)
            {
                this.connection.Dispose();
                this.connection = null;
            }
        }

    }

Key points:

• client declares anonymous queue (=with empty queue name) channel.QueueDeclare()
• server generates queue and returns queue name (amq.gen*)
• client adds queue name to message properties (props.ReplyTo = responseQueueName;)
• ServiceStack automatically sends response to temporary queue
• client picks up response and deserializes

It can be used like that:

using (var mqClient = new MqClient("mx.servicestack"))
{
    var pingResponse = mqClient.RpcCall<PingResponse>(new Ping { });
}

Important: You've got to use servicestack version 4.0.32+.