CODEWITHSUNDEEP
python-3.7aiohttp
user6037143
user6037143

Reputation: 566

aiohttp - concurrent requests are getting hung

Concurrent requests are getting hung. Here's a sample code that I am using to test concurrent requests.

import aiohttp
import asyncio

async def fetch(session, url):
    async with session.get(url) as response:
        print(await response.text())

async def main(n):
    url = "https://httpstat.us/200"
    async with aiohttp.ClientSession() as session:
        tasks = [asyncio.create_task(fetch(session, url)) for _ in range n]
        await asyncio.gather(*tasks)

asyncio.run(main(10))

When I make 10 concurrent requests, the first 4-5 requests are made concurrently and then it gets hung for over 10 seconds and then starts the remaining tasks which get hung again after running 2-3 concurrent requests. If I make 100 concurrent requests, it makes about 25-30 concurrent requests and gets hung and then makes 5-6 requests and gets hung again, it does this until all tasks are complete.

It's taking over two minutes to make 100 requests to https://httpstat.us/200 with aiohttp.

If I don't use a persistent ClientSession and create new ClientSession for every request, then all hundred requests finish within 5 seconds without getting hung.

I am not sure what I am doing here. Any help will be highly appreciated.

Upvotes: 4

Views: 968

Answers (1)

Doron Segal
Doron Segal

Reputation: 2260

I was able to run this code using: python 3.7, asyncio 3.4.3, aiohttp 3.5.4 

import aiohttp 
import asyncio 

async def fetch(session, url): 
    async with session.get(url) as response: 
        print(await response.text()) 

async def main(n): 
    url = "https://httpstat.us/200" 
    async with aiohttp.ClientSession() as session: 
        tasks = [asyncio.create_task(fetch(session, url)) for _ in range (0,n)] 
        await asyncio.gather(*tasks) 

asyncio.run(main(10))

Upvotes: 1

Related Questions