CODEWITHSUNDEEP
pythonmultithreadingpython-3.xpython-multithreadingpython-asyncio
Nyxynyx
Nyxynyx

Reputation: 63687

Getting asyncio to run a function in order (Python 3)

Here is a simple example of using asyncio to print out numbers from 0 to 9.

Problem: Sometimes the code prints out the numbers from 0 to 7, then prints 9, then 8. Especially when you set ThreadPoolExecutor to a smaller number like 4 or 5.

0
1
2
3
4
5
6
7
9
8

How do you get it to always print in sequence from 0 to 9? Why did it not print in sequence?

0
1
2
3
4
5
6
7
8
9

Code

import asyncio
from concurrent.futures import ThreadPoolExecutor


async def printThreaded(THREAD_POOL, length):   
    loop = asyncio.get_event_loop()
    futures = []
    for i in range(length):
        futures.append(loop.run_in_executor(THREAD_POOL, echo, i))
    await asyncio.wait(futures)


def echo(i):
    print(i)


THREAD_POOL = ThreadPoolExecutor(16)
with THREAD_POOL:
    loop = asyncio.get_event_loop()
    length = 10
    loop.run_until_complete(printThreaded(THREAD_POOL, length))

Upvotes: 1

Views: 1568

Answers (1)

Mikhail Gerasimov
Mikhail Gerasimov

Reputation: 39576

What happens in your code now?

You create list of coroutines (futures) that will each run echo in thread pool, than you start them all at once (await asyncio.wait(futures)). Since multiple echo running in same time and each prints on being run, all this prints can happen any time.

What do you want to do?

You probably don't really want to run coroutines in order (otherwise you can just call it in loop without asyncio), you want to run them in thread pool concurrently, but print their results in order coroutines were created

In this case you should:

  1. split actual work that will happen in thread from printing it

  2. probably prefer to use asyncio.gather to get computed results in order

  3. finally print ordered results you got in main thread

Summary

Here's modified version of your code that does explained above:

import time
from random import randint

import asyncio
from concurrent.futures import ThreadPoolExecutor


async def printThreaded(THREAD_POOL, length):   
    loop = asyncio.get_event_loop()

    # compute concurrently:
    coroutines = []
    for i in range(length):
        coroutine = loop.run_in_executor(THREAD_POOL, get_result, i)
        coroutines.append(coroutine)
    results = await asyncio.gather(*coroutines)

    # print ordered results:
    for res in results:
        print(res)



def get_result(i):

    time.sleep(randint(0, 2))  # actual work, reason you delegate 'get_result' function to threaed

    return i  # compute and return, but not print yet


THREAD_POOL = ThreadPoolExecutor(4)
with THREAD_POOL:
    loop = asyncio.get_event_loop()
    length = 10
    loop.run_until_complete(printThreaded(THREAD_POOL, length))

Upvotes: 3

Related Questions