CODEWITHSUNDEEP
queuepriority-queue
DatsunBing
DatsunBing

Reputation: 9076

Queues: How to process dependent jobs

I am working on an application where multiple clients will be writing to a queue (or queues), and multiple workers will be processing jobs off the queue. The problem is that in some cases, jobs are dependent on each other. By 'dependent', I mean they need to be processed in order.

This typically happens when an entity is created by the user, then deleted shortly after. Obviously I want the first job (i.e. the creation) to take place before the deletion. The problem is that creation can take a lot longer than deletion, so I can't guarantee that it will be complete before the deletion job commences.

I imagine that this type of problem is reasonably common with asynchronous processing. What strategies are there to deal with it? I know that I can assign priorities to queues to have some control over the processing order, but this is not good enough in this case. I need concrete guarantees.

Upvotes: 1

Views: 1271

Answers (2)

Andre Sardo
Andre Sardo

Reputation: 26

I think an handy feature for your use case is Job chaining:

https://laravel.com/docs/5.5/queues#job-chaining

Upvotes: 0

Mr Slim
Mr Slim

Reputation: 1458

This may not fit your model, but the model I have used involves not providing the deletion functionality until the creation functionality is complete.

When Create_XXX command is completed, it is responsible for raising an XXX_Created event, which also gets put on the queue. This event can then be handled to enable the deletion functionality, allowing the deletion of the newly created item.

The process of a Command completing, then raising an event which is handled and creates another Command is a common method of ensuring Commands get processed in the desired order.

Upvotes: 1

Related Questions