CODEWITHSUNDEEP
azureazureservicebus
Alex Kilpatrick
Alex Kilpatrick

Reputation: 411

Azure service bus dropping messages?

I am trying to build a simple Web API REST service in Azure with a service bus queue worker on the back end. I can send a single message from the Web API to the worker just fine. However, I was trying to send more messages just to see how everything works. So, I created a simple controller that looks like this:

for (int i = 0; i < 100; i++)
{
    var msg = new BrokeredMessage("Ping");
    BioConnector.QueueConnector.OrdersQueueClient.Send(msg);
}

When I call the controller, I am only getting about 1/2 or so of the messages being received by the worker. The rest seem to be dropped.

Upvotes: 3

Views: 6211

Answers (5)

user1069816
user1069816

Reputation: 2911

I had the same problem, sending one message worked fine but at peak times some messages were silently dropped.

The issue in my case was the default expiry set on the queue was too low so messages were expiring before the consumer was able to process them. By default these messages are dropped but the queue can be set to move them to the Dead Letter Queue, this makes troubleshooting lost messages easier.

enter image description here

Docs: https://learn.microsoft.com/en-gb/azure/service-bus-messaging/message-expiration#entity-level-expiration

Upvotes: 0

robinson3443
robinson3443

Reputation: 1

I had the WindowsAzure.ServiceBus NuGet package in my project and used QueueClient.Send() to send messages and faced the same messages lost issue.

My solution to completely solve the issue:

On the send side, I had to use REST API to send messages.

On the receive side, this is how I extracted the message body:

using (var stream = brokeredMessage.GetBody<Stream>())
{
    using (var streamReader = new StreamReader(stream, Encoding.UTF8))
    {
        var msg = streamReader.ReadToEnd();
        // Convert the JSON message to an object
        // var obj = JsonConvert.DeserializeObject<ObjectType>(msg);
    }
}

Upvotes: 0

Abhishek Lal
Abhishek Lal

Reputation: 3231

Azure Service Bus provides durable messaging so you will not loose any messages. Some items to investigate further: 1) Is there another instance of the worker role that is pulling off messages from that queue 2) are you using peek-lock as the receive mode as that will be the only way to guarantee at least once delivery. Receive and delete mode does not have the guarantee 3) are the messages going into dead-lettered queue due to either message expiry or exceeding max delivery count, I.e. They are received but not completed several times 4) if none of the above apply then raise a support ticket and the Azure product team can investigate the symptoms because as I mentioned this is a durable messaging system so no messages will be "lost".

Upvotes: 2

Alex Kilpatrick
Alex Kilpatrick

Reputation: 411

This was a weird problem. Through a random walk through "trying things" I ended up changing the string name of the queue and then everything started working again. I didn't change anything but the name of the queue - no changes to any configuration parameters at all.

It appears to be something buggy with that particular queue on Azure.

Upvotes: 2

Sean Osterberg
Sean Osterberg

Reputation: 844

I had issues with getting only about half the messages using the sample code posted here, so I wrote my own test code. I've tried it with > 100 queue messages and have always had 100% send/rec'd parity. Perhaps you had a similar issue with the code.

  1. Create a new C# console project.
  2. Add a reference to the Microsoft.ServiceBus assembly located in C:\Program Files\Microsoft SDKs\Windows Azure.NET SDK\2012-06\ref\Microsoft.ServiceBus.dll.

  3. In the app.config, change it to this with your own values provided:

    <appSettings>
    <add key="Microsoft.ServiceBus.ConnectionString" value="Endpoint=sb://blahblah.servicebus.windows.net/;SharedSecretIssuer=owner;SharedSecretValue=pDk0b....=" />
</appSettings>

  4. Add these using directives:

    using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using System.Xml;
using System.Xml.Linq;
using Microsoft.ServiceBus;
using Microsoft.ServiceBus.Messaging;
using System.Configuration;
using System.Threading;

  5. Change the code method to the following:

    class Program
{
    static void Main(string[] args)
    {
        string connectionString = ConfigurationSettings.AppSettings["Microsoft.ServiceBus.ConnectionString"];
        var namespaceManager = NamespaceManager.CreateFromConnectionString(connectionString);
        QueueDescription queueDesc = new QueueDescription("TestQueue");
        if (!namespaceManager.QueueExists(queueDesc.Path))
        {
            namespaceManager.CreateQueue(queueDesc);
        }
        QueueClient topicClient = QueueClient.CreateFromConnectionString(connectionString, queueDesc.Path);
        int sentMsgCount = 0;
        int recdMsgCount = 0;

        for (int i = 0; i < 100; i++)
        {
            BrokeredMessage msg = new BrokeredMessage("Test message " + i);
            topicClient.Send(msg);
            sentMsgCount++;
            Console.WriteLine("Sent Message: " + msg);
        }

        QueueClient subClient = QueueClient.CreateFromConnectionString(connectionString, queueDesc.Path);

        bool moreMessages = true;
        while (moreMessages)
        {
            BrokeredMessage recdMsg = subClient.Receive(TimeSpan.FromSeconds(3));
            if (recdMsg != null)
            {
                Console.WriteLine("Received Message: " + recdMsg);
                recdMsgCount++;
                recdMsg.Complete();
            }
            else
            {
                moreMessages = false;
            }
        }
        Console.WriteLine("# of sent msgs: " + sentMsgCount + ", # of rec'd msgs: " + recdMsgCount);
        Console.Read();
    }
}

Upvotes: 3

Related Questions