CODEWITHSUNDEEP
celerydjango-celery
JasonGenX
JasonGenX

Reputation: 5444

Django/Celery: max_retries works for re-try, but printed value is always the Celery default (3)

    @app.task(bind=True, autoretry_for=(Exception,), retry_kwargs={'max_retries': 10, 'countdown': 5})
    def job_deliver_message(self, message_id):
        print('Try {0}/{1}'.format(self.request.retries, self.max_retries))
        ...
        ...
        ...

To test, I'm causing this task to fail on purpose. it DOES re-try 10 times!, however, the printout is like:

[2020-06-03 19:14:06,700: WARNING/ForkPoolWorker-15] Try 0/3
[2020-06-03 19:14:12,080: WARNING/ForkPoolWorker-1] Try 1/3
[2020-06-03 19:14:17,553: WARNING/ForkPoolWorker-3] Try 2/3
[2020-06-03 19:14:23,000: WARNING/ForkPoolWorker-5] Try 3/3
[2020-06-03 19:14:28,489: WARNING/ForkPoolWorker-7] Try 4/3
[2020-06-03 19:14:33,603: WARNING/ForkPoolWorker-9] Try 5/3
[2020-06-03 19:14:39,038: WARNING/ForkPoolWorker-11] Try 6/3
[2020-06-03 19:14:44,525: WARNING/ForkPoolWorker-13] Try 7/3
[2020-06-03 19:14:49,688: WARNING/ForkPoolWorker-15] Try 8/3
[2020-06-03 19:14:54,985: WARNING/ForkPoolWorker-1] Try 9/3
[2020-06-03 19:14:54,985: WARNING/ForkPoolWorker-1] Try 10/3

What am I missing? am I printing the value for the max_retries from the wrong place?

and Yes: I do know i can just show the "10"... as I explicitly set it... but just wanted to know why it's giving me 3 when it's given 10 and actually acting upon it for the number of re-tries...

Upvotes: 3

Views: 1434

Answers (3)

Michael Ushakov
Michael Ushakov

Reputation: 2809

I' ve implemented retry slighthly differnt i.e. i have following celery task in one of my projects:

@app.task(bind=True, max_retries=4, default_retry_delay=3)
def get_external_sentence_translation(self, translation_request):
    """
       Passing translation data to translator
    :param self
    :param translation_request:
    :return:
    """

    url = settings.TRANSLATOR_BULK_TRANSLATION_RESOURCE
    try:
        session = requests.Session()
        result = session.post(url=url, data=translation_request)
        if result.status_code == 500:
            self.retry()
        result_json = result.json(encoding='utf-8')
        # print (u'json: ' + json.dumps(result_json))
        if 'status' not in result_json:
            message = 'Auto translation in get_external_sentence_translation json status is undefined'
            logger.warning(message)
            raise RuntimeError(message)
        if 'result' not in result_json and result_json['status'] is False:
            message = 'Auto translation in get_external_sentence_translation json status is False'
            logger.warning(message)
            raise RuntimeError(message)
        session.close()
        if 'result' in result_json:
            validation_result = JsonEncoder.from_json(result_json['result'])
            return validation_result # result_json[u'result']
            return result_json
    except Exception as e:
        logger.error('Raised exception in get_external_sentence_translation')
        logger.error(e)
        try:
            raise self.retry()
        except MaxRetriesExceededError:
            return BulkTranslationDto(status='False').to_json()

here i'm manually call retry() and catch MaxRetriesExceededError to stop retrying

Upvotes: 0

maximus
maximus

Reputation: 161

You are using self.max_retries which is set to 3 by default.

The retry method has another variable max_retries which is set to 10 in this case.

Code reference: task.py

Upvotes: 2

Matthew Hegarty
Matthew Hegarty

Reputation: 4306

am I printing the value for the max_retries from the wrong place?

I think this could be the source of the issue. Can you delve into the Celery source and / or set breakpoints?

You could try setting max_retries in a class-based task and see if you get a different result:

class BaseTaskWithRetry(Task):
    autoretry_for = (Exception,)
    retry_kwargs = {'max_retries': 5}

Upvotes: 0

Related Questions