CODEWITHSUNDEEP
laravelredislaravel-5.6laravel-horizon
Vish K
Vish K

Reputation: 127

Job always fails at laravel jobs Redis rate limiting

This is a follow up on Laravel - Running Jobs in Sequence

I decided to go with redis rate limit. Code is below

jobClass {

  protected $subscription;

  public function __construct(Subscription$subscription) {
        $this->subscription= $subscription;
    }
  public function handle() {
    Redis::funnel('mailingJob')->limit(1)->then(function () {
            // Job logic...
            (new Mailer($this->subscription))->send();

        }, function () {
            // Could not obtain lock...
            return $this->release(10);
        });
  }
}

And the controller code looks like.

<?php

namespace App\Http\Controllers;

use App\Http\Controllers\Controller;
use App\Models\Subscriptions;
class MailController extends Controller
{


    public function sendEmail() {
        Subscriptions::all()
        ->each(function($subscription) {
            SendMailJob::dispatch($subscription);
        });
    }
}

Now, when i run the queue, some of them works rest(around 90%) failed with the below error in horizon.

SendMailJob has been attempted too many times or run too long. The job may have previously timed out.

What am i missing? Please someone guide me to the right direction.

My goal is to run only one job of a type at one time.

Upvotes: 0

Views: 2982

Answers (2)

Josh
Josh

Reputation: 956

The Laravel documentation has a hint about this:

When using rate limiting, the number of attempts your job will need to run successfully can be hard to determine. Therefore, it is useful to combine rate limiting with time based attempts.

The core of the issue is, the job keeps failing until it can achieve a lock and run.

So I imagine that where you are running your queue worker, you are not setting the --tries flag high enough.

Although you could just set a very high --tries, it is not really scalable.

The best solution, as suggested in the documentation, would be to increase the number of tries as well as using time based attempts

You can also increase return $this->release(10); the release time here. That should have the job wait longer before trying to reacquire a lock, so will use up fewer tries!

Upvotes: 2

Travis Britz
Travis Britz

Reputation: 5552

[...] has been attempted too many times or run too long is an error that doesn't tell you why the job failed. It means some other exception has caused your job to fail every time it was attempted by the worker, and the worker has tried it the maximum number of times it was allowed to by your configuration. To understand why it's failing, check your laravel.log file for the exception that caused the job to fail.

In your case, since Mailer is contacting an external system it could be that the system you're connecting to is rate limiting you, or they could be having temporary connection problems or other service downtime. Again, there should be more detail in your log files.

Upvotes: 2

Related Questions