CODEWITHSUNDEEP
c#rabbitmqeasynetq
Dawood Awan
Dawood Awan

Reputation: 7338

Easynetq - Read multiple messages from a RabbitMQ

I have a solution which consists of 3 services, there are 2 queues to pass on messages from one service to the other, and it's using the EasyNetQ C# library to connect to RabbitMQ.

This is the outline of the architecture:

  1. Receive data from network TCP (do some processing) push to Queue A
  2. Read from Queue A (do some processing) push to Queue B
  3. Read from Queue B, send processed data to a 3rd party API

Services 1 and 2 are old services while 3 has been introduced recently.

The thing is that 1 and 2 handle one log at a time, so the publish/subscribe pattern makes sense i.e. service 1 receives one message and pushes this one message to Queue A for further processing, but the Service 3 should send as many messages as possible to the 3rd party API, as the API can accept upto 1000 messages int a single HTTP POST request.

I thought of introducing a timer for example Every X mins the timer will trigger an event to read upto 1000 messages from the Queue and send them via HTTP to the 3rd party API, but I can't seem to find a way to manually read N number of messages from the Queue.

At the moment I publish the message from Service 2 to Queue B like this:

_bus.Publish(myObject);

Then in the 3rd service I have a subscription

_bus.SubscribeAsync<XQueueDTO>("x_queue_processor", ProcessMessageAsync);

but this subscribe will always read one message at a time from the Queue.

So, what would be the best way to handle this?

Upvotes: 1

Views: 2110

Answers (1)

bumblebee
bumblebee

Reputation: 1841

You can alter the BasicQos settings to allow the consumer to read 1000 messages. If that's not solving your problem you can read messages one by one and when the count reaches to 1000 you can trigger the logic of Service 3.

Edit #1:enter image description here

Reference: https://www.rabbitmq.com/tutorials/tutorial-two-dotnet.html

Upvotes: 1

Related Questions