CODEWITHSUNDEEP
pythonmultithreadingpython-asynciocoroutine
Chan Kha Vu
Chan Kha Vu

Reputation: 10400

List of coroutines, wait until at least one yielded in Python

I never used coroutines before, so this question might seem trivial to you. I have several generators that yield some values, as follows:

@asyncio.coroutine
def generator_1():
    while True:
        # do something
        yield value_1

@asyncio.coroutine
def generator_2():
    while True:
        # do something
        yield value_2

I want to wait until at least one of them yielded something, i.e. I don't need all yield values, only one of them. Currently, I have followed examples I found on the web and did the following:

async def get_at_least_one():
    await asyncio.wait(generator_1(), generator_2(), return_when=asyncio.FIRST_COMPLETED)

which gives me an error:

TypeError: expected a list of futures, not generator

I understand where this error came from, but don't know the correct way to make this work. How can I achieve what I described above, i.e. wait until at least one yielded?

Upvotes: 0

Views: 1063

Answers (2)

user4815162342
user4815162342

Reputation: 154911

First, the correct way to invoke asyncio.wait is to give it a sequence of awaitable objects as the first argument. For example (note the square brackets):

await asyncio.wait([coro1(), coro2()], return_when=...)

Second, although asyncio uses generators internally, it is not a library for working with generators a la itertools. It just uses generators as one of the ways to implement suspendable coroutines, also known as generator-based coroutines. But those generators have to behave in very specific ways - for example, the values they yield are not observed by the awaiter, they are propagated all the way to the event loop. The awaiter observes the final value returned by the generator-based coroutine, which is meaningless for ordinary generators and was introduced specifically for coroutines.

Third, the asyncio.coroutine decorator and the generator-based coroutines are deprecated in favor of native coroutines defined with async def, which were introduced in Python 3.5 and should be used in all asyncio code. yield inside an async def is allowed, but it implements an asynchronous generator, i.e. a generator over an async source.

When working with actual generators, you don't need asyncio; instead, take a look at itertools and related modules. For this particular use, to extract the first element out of a number of generators, you can probably just use the next function:

def extract_first(*iters):
    for it in iters:
        try:
            return next(it)
        except StopIteration:
            pass
    raise ValueError("all iterators are empty")

Upvotes: 1

user2357112
user2357112

Reputation: 280465

It sounds like you've misunderstood how generators and coroutines work.

Coroutines are a form of cooperative multitasking. They cannot run simultaneously. Yielding is how a coroutine suspends execution, typically because it has to wait for something.

Yielding is not how coroutines communicate values to each other, and yielding does not mean a coroutine has any results ready. Coroutines will usually never see the values other coroutines yield. In fact, in most asyncio code, coroutines will never explicitly yield at all - all yielding will be performed as part of awaiting on something that yields.

Whatever you're trying to achieve, you're going about it the wrong way. Most likely, either you shouldn't actually be using asyncio, or you shouldn't actually be using yield.

Upvotes: 1

Related Questions