CODEWITHSUNDEEP
pythondjangocelerydjango-celerycelerybeat
Guanghao Yu
Guanghao Yu

Reputation: 1

Part of celery-beat periodic task not triggered

In django admin panel, I created about 1500 celery-beat periodic tasks, and 100 of them has the same crontab schedule 15 4 * 1 * (m/h/d/dM/MY) UTC(Minute(s)/Hour(s)/Day(s) Of The Week/Day(s) Of The Month/Month(s) Of The Year). All of them were enabled.

But at 04:15 on day-of-month 1, one of the periodic tasks was not sending due task to celery worker while all other tasks were sending. From the django admin panel, this periodic task's last_run_time is None, which indicates that it is not triggered.

I tried to configure the crontab schedule to 15 * * * * (m/h/d/dM/MY) UTC and then it runs successfully at minute 15. So I wonder is there any limitation of the number of celery-beat periodic tasks?

celery.py

app = Celery('myapp', broker=os.getenv('BROKER_URL', None))

@signals.setup_logging.connect
def setup_logging(**kwargs):
    """Setup logging."""
    pass

app.conf.ONCE = {
  'backend': 'celery_once.backends.Redis',
  'settings': {
    'url': 'redis://localhost:6379/0',
    'default_timeout': 60 * 60
  }
}

app.autodiscover_tasks(lambda: settings.INSTALLED_APPS)

from django.utils import timezone
app.conf.update(
    CELERY_ALWAYS_EAGER=bool(os.getenv('CELERY_ALWAYS_EAGER', False)),
    CELERY_DISABLE_RATE_LIMITS=True,
    # CELERYD_MAX_TASKS_PER_CHILD=5,
    CELERY_TASK_RESULT_EXPIRES=3600,
    # Uncomment below if you want to use django-celery backend
    # CELERY_RESULT_BACKEND='djcelery.backends.database:DatabaseBackend',
    CELERY_ACCEPT_CONTENT=['json'],
    CELERY_TASK_SERIALIZER='json',
    CELERY_RESULT_SERIALIZER='json',
    # Uncomment below if you want to store the periodic task information in django-celery backend
    CELERYBEAT_SCHEDULER = "django_celery_beat.schedulers.DatabaseScheduler",
    # periodic tasks setup
    CELERYBEAT_SCHEDULE={
        'check_alerts_task': {
            'task': 'devices.tasks.alert_task',
            'schedule': 300.0  # run every 5 minutes
        },
        'weather_request': {
            'task': 'organizations.tasks.weather_request',
            'schedule': 60.0  # run every minute
        },
        'update_crontab': {
            'task': 'devices.tasks.update_crontab',
            'schedule': crontab(hour=0, minute=0)
        },
        'export_data_cleanup': {
            'task': 'devices.tasks.delete_expiry_export_data',
            'schedule': crontab(minute=0)
        }
    }
)

celery.log

[01/Jun/2020 04:15:09] INFO [celery.beat:271] Scheduler: Sending due task Report_first (reportAPI.tasks.report)

Upvotes: 0

Views: 930

Answers (1)

iklinac
iklinac

Reputation: 15748

Your crontab is actually wrong ( minute hour day month weekday )

15 4 * 1 * -> 04:15 each day in 1st month ( January )

what you want is 1st day of month 15 4 1 * *

Upvotes: 1

Related Questions