CODEWITHSUNDEEP
pythonrabbitmqceleryfastapi
Edward Wang
Edward Wang

Reputation: 51

How to get estimated time needed to finish Celery task?

I'm trying to get the estimated time needed to complete a Celery task. For example, if Task A is triggered on the frontend by a user (maybe with a click of a button that triggers a REST API call to the backend), it will display a timer to show how long more does it take for the task to finish.

If I did read Celery's documentation correctly, I can get the ETA for scheduled tasks. But what about the current task that has just been executed?

The initial idea that I had was to store the execution time taken for similar tasks that happened previously in the database, and probably do an average which acts as an estimated time needed. Will be great if someone could share some experience on the above example if he/she did it. Many thanks!

Upvotes: 2

Views: 1401

Answers (1)

Rupsi Kaushik
Rupsi Kaushik

Reputation: 41

Method 1: Use a timing decorator

import time
import functools


def timer(func):
    @functools.wraps(func)
    def _wrapper(*args, **kwargs):
        start = time.perf_counter()
        val = func(*args, **kwargs)
        end = time.perf_counter()
        elapsed_time = end - start
        task_info = {"time": elapsed_time, task_args: args, task_kwargs: kwargs}
        # do something with this task_info, i.e log or save in db to be used for analysis later
        return val

    return _wrapper


@app.task(name="task1")
@timer
def task1(...):
  ...

Method 2: Use celery signals

import time
from celery.signals import task_prerun, task_postrun

tasks = {}


@task_prerun.connect
def task_prerun_handler(task_id, task, **extras):
    """ Dispatched before a task is executed. """
    tasks[task_id] = time.perf_counter()


@task_postrun.connect
def task_postrun_handler(task_id, task, **extras):
    """ Dispatched after a task has been executed. """
    end = time.perf_counter()
    elapsed_time = end - tasks.pop(task_id)
    # do something with this task_info, i.e log or save in db to be used for analysis later

Resources you can look at:

Upvotes: 2

Related Questions