CODEWITHSUNDEEP
pythonpython-3.xlist-comprehensionpython-asyncio
Javier Levio
Javier Levio

Reputation: 111

Why am I getting "asynchronous comprehension outside of an asynchronous function"?

I'm using Python 3.7.4 and this block of code (MWE):

import asyncio


async def foo(x):
    await asyncio.sleep(1)
    return [i for i in range(10)]


async def big_foo():
    dict_of_lists = {'A': [i for i in range(10)], 'B': [i for i in range(10)]}
    return {
        key: [
            item
            for list_item in dict_of_lists[key]
            for item in await foo(list_item)
        ]
        for key in ['A', 'B']
    }

Throws the error:

File "filename.py", line 13
    item
    ^
SyntaxError: asynchronous comprehension outside of an asynchronous function

The error message doesn't help a lot because it is indeed inside an asynchronous function.

I solved the issue by defining explicit for loops instead of a comprehension, like this:

async def big_foo():
    dict_of_lists = {'A': [i for i in range(10)], 'B': [i for i in range(10)]}
    var = {}
    for key in ['A', 'B']:
        var[key] = []
        for list_item in dict_of_lists[key]:
            for item in await foo(list_item):
                var[key].append(item)
    return var

It bugs me that removing the dictionary outer loop also removes the error.

This works (it doesn't accomplish what I need obviously):

async def other_foo():
    dict_of_lists = {'A': [i for i in range(10)]}
    return [
        item
        for list_item in dict_of_lists['A']
        for item in await foo(list_item)
    ]

Upvotes: 4

Views: 2993

Answers (2)

crl_qjx
crl_qjx

Reputation: 93

This bug should have been fixed in Python 3.11.0

Asynchronous comprehensions are now allowed inside comprehensions in asynchronous functions. Outer comprehensions implicitly become asynchronous in this case. (Contributed by Serhiy Storchaka in bpo-33346.)

Upvotes: 1

Javier Levio
Javier Levio

Reputation: 111

I have found the answer. Quoting a co-worker:

The interior comprehension is asynchronous. The exterior one isn't. The function is asynchronous. The error is triggered because you are defining as an asynchronous comprehension inside of a non-asynchronous context - the error message is indeed wrong and it's a know bug.

https://bugs.python.org/issue33346

I fixed it by setting an async for in the exterior comprehension. Like this:

async def big_foo():
    dict_of_lists = {'A': [i for i in range(10)], 'B': [i for i in range(10)]}
    return {
        key: [
            item
            for list_item in dict_of_lists[key]
            for item in await foo(list_item)
        ]
        async for key in ['A', 'B']
    }

While this removes the mentioned SyntaxError it introduces a TypeError: 'async for' requires an object with __aiter__ method, got tuple. So the solution would be to define a wraper for it, but it looks more like a hack than a proper solution.

Upvotes: 5

Related Questions