CODEWITHSUNDEEP
pythonmultithreadingpython-asyncioaiohttp
lorenzocastillo
lorenzocastillo

Reputation: 1025

Does Python asyncio use a thread pool?

I wrote this bit of code:

import asyncio
import threading
from aiohttp import ClientSession

async def fetch(url):
    async with ClientSession() as session:
        async with session.get(url) as response:
            response = await response.read()
            print(threading.current_thread().name)


loop = asyncio.get_event_loop()

tasks = [asyncio.ensure_future(fetch("http://example.com")) for i in range(5)]

loop.run_until_complete(asyncio.wait(tasks))

It prints out "MainThread" every time. Does this mean that all of the requests are being executed concurrently using the same main thread and it's not using threads from a thread pool to execute each request or are the threads being abstracted?

This post seems to show that in fact there is a thread pool which Python uses for these async tasks: http://skipperkongen.dk/2016/09/09/easy-parallel-http-requests-with-python-and-asyncio/

Upvotes: 20

Views: 15031

Answers (1)

wim
wim

Reputation: 362587

It prints out "MainThread" every time. Does this mean that all of the requests are being executed concurrently using the same main thread and it's not using threads from a thread pool to execute each request or are the threads being abstracted?

It's not using worker threads, asyncio will only use the main thread. Concurrency is achieved through cooperative multitasking by using Python generators (read about coroutines).

This post seems to show that in fact there is a thread pool ...

As well as asyncio module, the blog post you linked explicitly uses the concurrent.futures module. The ThreadPoolExecutor class from that code will spawn worker threads. But the example code in your question will not.

Upvotes: 22

Related Questions