CODEWITHSUNDEEP
pythonpython-3.xasync-awaitpython-asynciocoroutine
Yuval Pruss
Yuval Pruss

Reputation: 9826

python asyncio add_done_callback with async def

I have 2 functions: The first one, def_a, is an asynchronous function and the second one is def_b which is a regular function and called with the result of def_a as a callback with the add_done_callback function.

My code looks like this:

import asyncio

def def_b(result):
    next_number = result.result()
    # some work on the next_number
    print(next_number + 1)

async def def_a(number):
    await some_async_work(number)
    return number + 1

loop = asyncio.get_event_loop()
task = asyncio.ensure_future(def_a(1))
task.add_done_callback(def_b)
response = loop.run_until_complete(task)
loop.close()

And it's work perfectly.

The problem began when also the second function, def_b, became asynchronous. Now it looks like this:

async def def_b(result):
    next_number = result.result()
    # some asynchronous work on the next_number
    print(next_number + 1)

But now I can not provide it to the add_done_callback function, because it's not a regular function.

My question is- Is it possible and how can I provide def_b to the add_done_callback function if def_b is asynchronous?

Upvotes: 27

Views: 33592

Answers (4)

Mohsen HNSJ
Mohsen HNSJ

Reputation: 13

I'm on python 3.12 and found this easy solution with least added complexity:

   # We define a regular non async function that accepts a context of type asyncio.Task
   # You can omit this, if you don't need the result of the previously completed task
   # The context holds valuable data like "status" and "result" of completed task
   def my_callback(context: asyncio.Task):
      """Starts the specified async callback functions"""
      
      # Using asyncio, we create a single task that starts our intended callback function
      # which is async, and we pass the context to it for accessing completed task result
      asyncio.create_task(task_finished(context))

and you call this function, instead of your actual function you need to run after the task completion:

   ...
   # You call the newly created non async function
   # This non async function, will start your intended async function
   # The context will be passed automatically
   very_long_task.add_done_callback(my_callback)
   ...

By the way, credit goes to Jason Brownlee who originally posted this solution on superfastpython.com

Upvotes: -1

Udi
Udi

Reputation: 30472

add_done_callback is considered a "low level" interface. When working with coroutines, you can chain them in many ways, for example:

import asyncio


async def my_callback(result):
    print("my_callback got:", result)
    return "My return value is ignored"


async def coro(number):
    await asyncio.sleep(number)
    return number + 1


async def add_success_callback(fut, callback):
    result = await fut
    await callback(result)
    return result


loop = asyncio.get_event_loop()
task = asyncio.ensure_future(coro(1))
task = add_success_callback(task, my_callback)
response = loop.run_until_complete(task)
print("response:", response)
loop.close()

Keep in mind add_done_callback will still call the callback if your future raises an exception (but calling result.result() will raise it).

Upvotes: 35

J. Doe
J. Doe

Reputation: 447

You can try the aiodag library. It's a very lightweight wrapper around asyncio that abstracts away some of the async plumbing that you usually have to think about. From this example you won't be able to tell that things are running asynchronously since it's just 1 task that depends on another, but it is all running async.

import asyncio
from aiodag import task

@task
async def def_b(result):
    # some asynchronous work on the next_number
    print(result + 1)

@task
async def def_a(number):
    await asyncio.sleep(number)
    return number + 1

async def main():
    a = def_a(1)
    b = def_b(a)  # this makes task b depend on task a
    return await b

loop = asyncio.get_event_loop()
asyncio.set_event_loop(loop)
response = loop.run_until_complete(main())

Upvotes: 1

user3593261
user3593261

Reputation: 568

This only works for one future job, if you have multiple async jobs, they will blocks each other, a better way is using asyncio.as_completed() to iterate future list:

import asyncio

async def __after_done_callback(future_result):
    # await for something...
    pass

async def __future_job(number):
    await some_async_work(number)
    return number + 1

loop = asyncio.get_event_loop()
tasks = [asyncio.ensure_future(__future_job(x)) for x in range(100)]  # create 100 future jobs

for f in asyncio.as_completed(tasks):
    result = await f
    await __after_done_callback(result)

loop.close()

Upvotes: 6

Related Questions