CODEWITHSUNDEEP
node.jsqueuejobs
pauljeba
pauljeba

Reputation: 770

Conditional Retry of Failed Jobs using Bull in Node.js

I know we can retry failed jobs in Bull using the backoff attribute in Bull like below

var Queue = require('bull');
var myQueue = new Queue('myQueue');

runQueryQueue
    .add(data, {
      jobId: uid,
      attempts: 10,
      backoff: {
        type: 'exponential', 
        delay: 2000, 
      },
      stackTraceLimit: 100,
    })
    .then((result) => {})
    .catch(console.log);

But I want to retry only when the failure reasons are worth retrying - example timeouts, rate limits, etc and not retry when the errors are related to user inputs, etc.

How can I check the failure error message and decide to retry?

Upvotes: 1

Views: 3618

Answers (4)

Tom VanAntwerp
Tom VanAntwerp

Reputation: 11

bull has a job.discard() function that should allow you to mark a job as non-retriable in the process.

Upvotes: 1

Lem Ko
Lem Ko

Reputation: 423

This answer is for bull-mq so would be useful if you're willing to upgrade.

There is now an UnrecoverableError you can throw and bull-mq will take care of not retying the job.

https://docs.bullmq.io/guide/retrying-failing-jobs#stop-retrying-jobs

Upvotes: 0

Virinum
Virinum

Reputation: 31

Keep setting your backoff config. If the job is not worth retrying, set opts.attempts to attemptsMade + 1 or smaller inside the processor.

Your changed opts will not be saved, but your job will be moved to the failed list instantly.

async process(job: Job): Promise<void> {
  try {
    await doStuff(job);
  } catch (error) {
    if (!isWorthRetrying(error)) {
      // Move job to failed list instantly by setting attemps to attemptsMade + 1 or smaller
      job.opts.attempts = job.attemptsMade + 1;
    }
    throw error;
  }
}

Upvotes: 2

Animir
Animir

Reputation: 1204

You can mark a job as complete, e.g. do not throw errors, but write error data to a result of job completion. Result can be processed different way, if there is any sign of error.

async function jobProcess(job) {
  if (doNotRetryError) {
    return doNotRetryError
  } else if (anyOtherError) {
    throw new Error('retry')
  } else {
    return {success: true}
  }
}

async function jobCompleted(job, result) {
  if (result instanceof Error) {
    // some error happened, but job shouldn't be retried
  } else {
    // job completed
  }
}

Upvotes: 1

Related Questions