CODEWITHSUNDEEP
pythonasync-awaitaiohttp
DCorrigan
DCorrigan

Reputation: 666

multiple nonblocking tasks using asyncio and aiohttp

I am trying to perform several non blocking tasks with asyncio and aiohttp and I don't think the way I am doing it is efficient. I think it would be best to use await instead of yield. can anyone help?

def_init__(self):
    self.event_loop = asyncio.get_event_loop()

def run(self):
    tasks = [
        asyncio.ensure_future(self.subscribe()),
        asyncio.ensure_future(self.getServer()),]
    self.event_loop.run_until_complete(asyncio.gather(*tasks))
    try:
       self.event_loop.run_forever()

@asyncio.coroutine
def getServer(self):
    server = yield from self.event_loop.create_server(handler, ip, port)
    return server

@asyncio.coroutine
def sunbscribe(self):
    while True:
        yield from asyncio.sleep(10)
        self.sendNotification(self.sub.recieve())

def sendNotification(msg):
    # send message as a client

I have to listen to a server and subscribe to listen to broadcasts and depending on the broadcasted message POST to a different server.

Upvotes: 6

Views: 1161

Answers (2)

Maresh
Maresh

Reputation: 4712

According to the PEP 492:

await , similarly to yield from , suspends execution of read_data coroutine until db.fetch awaitable completes and returns the result data.

It uses the yield from implementation with an extra step of validating its argument. await only accepts an awaitable , which can be one of:

So I don't see an efficiency problem in your code, as they use the same implementation.

However, I do wonder why you return the server but never use it.

The main design mistake I see in your code is that you use both:

self.event_loop.run_until_complete(asyncio.gather(*tasks))
try:
   self.event_loop.run_forever()

From what I can see you just need the run_forever()

Some extra tips:

In my implementations using asyncio I usually make sure that the loop is closed in case of error, or this can cause a massive leak depending on your app type.

try:
    loop.run_until_complete(asyncio.gather(*tasks))
finally:  # close the loop no matter what or you leak FDs
    loop.close()

I also use Uvloop instead of the builtin one, according to benchmarks it's much more efficient.

import uvloop
...
    loop = uvloop.new_event_loop()
    asyncio.set_event_loop(loop)

Upvotes: 2

Sam Hartman
Sam Hartman

Reputation: 6489

Await will not be more efficient than yield from. It may be more pythonic, but

async def foo():
    await some_future

and

@asyncio.coroutine
def foo()
    yield from some_future

are approximately the same. Certainly in terms of efficiency, they are very close. Await is implemented using logic very similar to yield from. (There's an additional method call to await involved, but that is typically lost in the noise)

In terms of efficiency, removing the explicit sleep and polling in your subscribe method seems like the primary target in this design. Rather than sleeping for a fixed period of time it would be better to get a future that indicates when the receive call will succeed and only running subscribe's task when receive has data.

Upvotes: 0

Related Questions