CODEWITHSUNDEEP
microservicesmessage-queuemessaging
Matt Ghafouri
Matt Ghafouri

Reputation: 1587

How to being ensure from putting message into a Message Queue?

I have set up a RabbitMQ for messaging, the integration works fine. I am sending messages to a queue and consuming from the queue. but how can I be sure about sending a message to the queue after performing a task?

For example, After I Add a new item into DataBase, I should publish a message to the queue.

DBContext.SaveChanges(item);

//what if an error occurred at this moment.
QueueManager.Publish(item)

What is the best practice in this situation? any suggestion will appreciate.

I read this solution Solution.

But I think there should be a better solution because there are two problems with this approach:

  1. in microservice architecture may some of the databases do not support atomic transaction
  2. it needs extra data manipulation and an extra background service for removing records from databases

Upvotes: 3

Views: 1341

Answers (4)

user7571491
user7571491

Reputation:

in microservice architecture may some of the databases do not support atomic transaction

I would think if you can be in position where publishing can fail, and the functionality is critical, then atomic transaction support cannot be optional. So, you try to publish, if it fails, rollback transaction. That means in the solution Best way to ensure an event is eventually published to a message queuing sytem, I would not even prefer to use async publish.

If the functionality is not critical then you can do all sort of optimization, including putting error messages in logs in case it fails to publish once in a while.

it needs extra data manipulation and an extra background service for removing records from databases

Normally, users prefer safe than sorry approach. So this should be okay when functionality is not critical.

Upvotes: 1

Jonathazn
Jonathazn

Reputation: 11

I don't know if I'm misinterpreting your question or the other people who answered your question have, but as I understand it, you want to publish a message to the queue if SaveChanges() has finished succesfully? Normally the method should return a boolean or the object you're saving so if that returns true/the object you can send the message.

Something like this:

if(DBContext.SaveChanges(item)) {
QueueManager.Publish(item)
} else {
// Error handling

}

Upvotes: 0

Vaibs
Vaibs

Reputation: 1606

how can I be sure about sending a message to the queue after performing a task?

Create the new table name Event, use the Event table which contains record of NonPublished events. Once done with the Database call, make an entry in the event table and mark that event as a NonPublished. And after publishing the event to queue, update that record as a Published event.

What happens if the application not able to publish the event(Queue system is down, the publisher is failed)?

Write a background job/cron job that runs periodically, which will check all Unpublished events and publish to queue and make the status as a Published. Also configure the retry mechanism, as mentioned by Tarun if the event published failed.

Please refer to the below link for more information about Event-driven microservices.

https://www.nginx.com/blog/event-driven-data-management-microservices/

Upvotes: 3

Tarun
Tarun

Reputation: 986

You can use a retry pattern from Resiliency patterns of Microservices.

Retry – The source application can immediately retry to send the request to the service. If the specific fault is unusual or rare, the probability of success when repeating the request is very high.

Retry after a delay – The source application can retry to send the request to the cloud service after a period of time (that normally increases exponentially). This is a common practice when the failure event is due to reasons such as cloud service busy and so on.If the fault is caused by one of the more commonplace connectivity or busy failures, then the application has to wait for some time and try again.

Source : Retry Pattern

Upvotes: 0

Related Questions