CODEWITHSUNDEEP
pythonpython-asyncioaiohttp
ozn
ozn

Reputation: 2238

asyncio + aiohttp : why is my client still a blocking?

I'm trying to test my python 3.4.3 http client or application that sends HTTP requests to a server. If the server to have a delay to respond for whatever reason, I'm expecting the requests should not be blocking as asyncio+aiohttp should provide asynchronous calls:

def post(self):
    print("received post")
    print(self.request)
    print("POST Body: %s" % str(self.request.body))
    time.sleep(3)
    self.write("blah")
    self.finish()

I'm just wondering why my code/http client is blocking:

import aiohttp, asyncio, async_timeout

@asyncio.coroutine
def fetch(session, url):
    with aiohttp.Timeout(30):
        try:
            response = yield from session.get(url)
            print((yield from response.read()))
            return response
        except Exception as e:
            raise e
        finally:
            try:
                response.release()
            except:
                pass

@asyncio.coroutine
def post(session, url):
    with aiohttp.Timeout(30):
        try:
            response = yield from session.post(url)
            print((yield from response.read()))
            return response
        except Exception as e:
            raise e
        finally:
            try:
                response.release()
            except:
                pass

@asyncio.coroutine
def close_connection(session):
    try:
        session.close()
    except:
        pass

if __name__ == '__main__':
    loop = asyncio.get_event_loop()
    session = aiohttp.ClientSession(loop=loop)
    try:
        for i in range(10):
            html = loop.run_until_complete(post(session, 'http://localhost:8000'))
    except Exception as e:
        print("received exception %s." % type(e).__name__)

        # Close session if not closed.
    loop.run_until_complete(close_connection(session))

I tried to do a loop where i iterate through 10 URLs (its the same one here). If it were sequential i would expect it to take ~30 seconds (server responds with 3 second delay). With asynchronous, my expectation is that it takes less.

try:
    for i in range(10):
        html = loop.run_until_complete(post(session, 'http://localhost:8000'))

Is it the "run_until_complete()" function that is blocking? How do I make it non-blocking?

Upvotes: 0

Views: 2570

Answers (1)

caramel1995
caramel1995

Reputation: 3055

Basically, when you use the run_until_complete(), you are telling the event loop to run the coroutine(s) passed as argument and return the result upon completion. In short, run_until_complete() will block until it finishes that operation.

Based on your for loop code snippet. Basically in every loop, the run_until_complete will block, run the coroutine (post in this context), return the result, then only the execution will continue for the next loop.

What you wanted to do here is to run all the post concurrently. What you can do is to use asyncio.gather().

try:
    html = loop.run_until_complete(asyncio.gather(*[post(session, 'http://localhost:8000') for i in range(10)]))

Upvotes: 3

Related Questions