CODEWITHSUNDEEP
phplaravel
Felix
Felix

Reputation: 2661

Laravel - Task Scheduling or Queue?

In my Laravel application, users can upvote or downvote on submissions.

To calculate the total score of a submission, I don't do it on every vote because this would cause a MASSIVE amount of queries.

Instead, every few minutes, I run a task with Laravel's task scheduler take all the recent submissions and then calculates the total score of each one.

Task Command:

$submissions = Submission::where('created_at', '>', Carbon::now()->subDays(2))
    ->withCount('upVotes')
    ->withCount('downVotes')
    ->get();

$submissions->each(function ($item) {
    $item->total_upvotes = $item->up_votes_count;
    $item->total_downvotes = $item->down_votes_count;
    $item->total_score = $this->score($item->up_votes_count, $item->down_votes_count);
    $item->hotness = $this->hotness($item->up_votes_count, $item->down_votes_count, $item->created_at->timestamp);
    $item->save();
});

However, I worry that this might not work very well when scaled to potentially thousands of submissions a day. Maybe this task would time out?

Alternatively, instead of doing the foreach in the task scheduler command, I could have the command dispatch a job that does the foreach, like so:

Task Command:

$submissions = Submission::where('created_at', '>', Carbon::now()->subDays(2))
    ->withCount('upVotes')
    ->withCount('downVotes')
       ->get();

//DISPATCH QUEUE JOB

Queue Job:

$submissions->each(function ($item) {
    $item->total_upvotes = $item->up_votes_count;
    $item->total_downvotes = $item->down_votes_count;
    $item->total_score = $this->score($item->up_votes_count, $item->down_votes_count);
    $item->hotness = $this->hotness($item->up_votes_count, $item->down_votes_count, $item->created_at->timestamp);
    $item->save();
});

Would this be better?

Upvotes: 1

Views: 483

Answers (1)

Tarps
Tarps

Reputation: 1933

Yes - that would be better.

I have a project with a similar workload where I download thousands of products from an external warehouse API and go through each one to update prices, images and descriptions of live products. This is also something that takes a very long time when done in a single for-loop and a single job definitely wouldn't suffice.

I use Laravel Horizon to handle my queues and have separate queues for a lot of different actions (processing, mailing etc). The data processing queue runs up to 10 of the jobs at the same time and that makes it even faster than the single for-loop job.

Upvotes: 1

Related Questions