CODEWITHSUNDEEP
celery
eugene
eugene

Reputation: 41665

Can I use celery for every-second task?

I'm running a task every second, and it seems celery doesn't actually perform the task every second.

I guess celery might be a good scheduler for every 1 minute task, but might not be adequte for every second task.

Here's the picture which illustrates what I mean.

enter image description here

I'm using the following options

     'schedule': 1.0,
     'args': [],
     'options': {
         'expires': 3
     }

And I'm using celery 4.0.0

Upvotes: 2

Views: 8424

Answers (1)

codyc4321
codyc4321

Reputation: 9672

Yes, Celery actually handles times as low as 1 second, and possibly lower since it takes a float. See this entry of periodic tasks in the docs http://docs.celeryproject.org/en/latest/userguide/periodic-tasks.html:

from celery import Celery
from celery.schedules import crontab

app = Celery()

@app.on_after_configure.connect
def setup_periodic_tasks(sender, **kwargs):
    # Calls test('hello') every 10 seconds.
    sender.add_periodic_task(10.0, test.s('hello'), name='add every 10')

    # Calls test('world') every 30 seconds
    sender.add_periodic_task(30.0, test.s('world'), expires=10)

    # Executes every Monday morning at 7:30 a.m.
    sender.add_periodic_task(
        crontab(hour=7, minute=30, day_of_week=1),
        test.s('Happy Mondays!'),
    )

@app.task
def test(arg):
    print(arg)

A better written example can be found 1/3 the way down https://github.com/celery/celery/issues/3589:

# file: tasks.py
from celery import Celery

celery = Celery('tasks', broker='pyamqp://guest@localhost//')

@celery.task
def add(x, y):
    return x + y


@celery.on_after_configure.connect
def add_periodic(**kwargs):
    celery.add_periodic_task(10.0, add.s(2,3), name='add every 10')

So sender is the actual Celery broker, i.e. app = Celery()

Upvotes: 6

Related Questions