CODEWITHSUNDEEP
pythonpython-3.xmultithreadingasynchronouspython-asyncio
dereks
dereks

Reputation: 564

Execute ThreadPoolExecutor asynchronously but use only 1 process and 1 thread

Can I limit ThreadPoolExecutor to use only 1 processor core, 1 process and 1 thread at the same time but run a function asynchronously?

import time, concurrent.futures

def ts(start):
    return round(time.perf_counter() - start, 3)

def fn(sec):
    print(f'{ts(start)}\tstart sleeping {sec}')
    time.sleep(sec)

def main():
    lst = [1, 1, 1, 1, 1, 1]
    with concurrent.futures.ThreadPoolExecutor(max_workers=1) as executor:
        executor.map(fn, lst)

start = time.perf_counter()

if __name__ == "__main__":
    main()
    print(f'{ts(start)}\tfinish')

# 0.001   start sleeping 1
# 1.008   start sleeping 1
# 2.01    start sleeping 1
# 3.021   start sleeping 1
# 4.024   start sleeping 1
# 5.038   start sleeping 1
# 6.051   finish

I'd like to run all the functions successively but instead of waiting the result after each run, wait all the results at the end like this:

# 0.001   start sleeping 1
# 0.002   start sleeping 1
# 0.003   start sleeping 1
# 0.004   start sleeping 1
# 0.005   start sleeping 1
# 0.006   start sleeping 1
# 1.051   finish

Upvotes: 0

Views: 2078

Answers (1)

alex_noname
alex_noname

Reputation: 32293

Thread pool works as follows, it starts a task in a thread and until it returns control the thread is busy, even if it sleeps. Therefore, if you want to run them in parallel, you have to either use multiple threads or use asyncio coroutines.

Here is asyncio sample:

import asyncio
import time

def foo():
    print('Start ...')
    time.sleep(5)
    print('... End')

async def main():
    loop = asyncio.get_running_loop()
    count = range(2)
    futures = [loop.run_in_executor(None, foo) for i in count]
    results = await asyncio.gather(*futures)

asyncio.run(main())

Output

Start ...
Start ...
... End
... End

Upvotes: 1

Related Questions