CODEWITHSUNDEEP
laravelqueuejobsbeanstalkd
Gordon Freeman
Gordon Freeman

Reputation: 3421

Laravel: sequential queue with beanstalkd

The project:

Users / Customers can upload a PDF - for reasons. This can happen simultanously.

The goal:

A large PDF (with many pages) must be converted. While converting the pages every individual step (4 different steps for every page) must get logged, so i can show the progress of the current step and calculate the processed percentage.

The situation:

After I get the amount of pages in the uploaded pdf, I run a for loop and push the convert job into the beanstalkd queue. Everything works fine.

The problem:

The pages getting processed in a different order than they came in through the loop. This behaviour results in confusing log entries and can not be used to display the progress.

beanstalkd config

[program:worker]
process_name=%(program_name)s_%(process_num)02d
command=php artisan queue:work --sleep=3 --tries=3 --daemon
directory=/home/vagrant/Code/project
autostart=true
autorestart=true
numprocs=8
redirect_stderr=true

I've set numprocs to 8 so that multiple users can upload their files simultanously, but it results un unwanted chaotic order.

If I change numprocs to 1 it runs in correct order but than I can not run multiple converting processes at once.

What should I do?

EDIT

I'm so dumb. Just solved the problem. But for the glory of satan (and so someone can earn the points) I'll post the solution tomorrow.

Upvotes: 1

Views: 381

Answers (1)

Gordon Freeman
Gordon Freeman

Reputation: 3421

Since nobody answered; here is the solution:

instead of running a for loop and pushing all the page converting jobs into the queue, I just push a single job into the queue which converts one page and creates a new job for the next page (if there is one) after it's done.

Upvotes: 1

Related Questions