CODEWITHSUNDEEP
pythonyield
Lufftre
Lufftre

Reputation: 130

Misuse of yield

I'm making a SocketServer that will need to be able to handle a lot of commands. So to keep my RequestHandler from becoming too long it will call different functions depening on the command. My dilemma is how to make it send info back to the client.

Currently I'm making the functions "yield" everything it wants to send back to the client. But I'm thinking it's probably not the pythonic way.

# RequestHandler
func = __commands__.get(command, unkown_command)
for message in func():
    self.send(message)


# example_func
def example():
    yield 'ip: {}'.format(ip)
    yield 'count: {}'.format(count)

    . . .

    for ping in pinger(ip,count):
        yield ping

Is this an ugly use of yield? The only alterative I can think of is if when the RequestHandler calls the function is passes itself as an argument

func(self)

and then in the function

def example(handler):
    . . .
    handler.send('ip: {}'.format(ip))

But this way doesn't feel much better.

Upvotes: 1

Views: 144

Answers (2)

Goswin von Brederlow
Goswin von Brederlow

Reputation: 12322

I do the same as you with yield. The reason for this is simple:

With yield the main loop can easily handle the case that sending data to one socket will block. Each socket gets a buffer for outgoing data that you fill with the yield. The main loop tries to send as much of that as possible to the socket and when it blocks it records how far it got in the buffer and waits for the socket to be ready for more. When the buffer is empty is runs next(func) to get the next chunk of data.

I don't see how you would do that with handler.send('ip: {}'.format(ip)). When that socket blocks you are stuck. You can't pause that send and handle other sockets easily.

Now for this to be useful there are some assumptions:

  • the data each yield sends is considerable and you don't want to generate all of it into one massive buffer ahead of time

  • generating the data for each yield takes time and you want to already send the finished parts

  • you want to use reply = yield data waiting for the peer to respond to the data in some way. Yes, you can make this a back and forth. next(func) becomes func.send(reply).

Any of these is a good reason to go the yield way or coroutines in general. The alternative seems to be to use one thread per socket.

Note: the func can also call other generators using yield from. Makes it easy to split a large problem into smaller handlers and to share common parts.

Upvotes: 0

loopbackbee
loopbackbee

Reputation: 23322

def example():
    yield 'ip: {}'.format(ip)
    yield 'count: {}'.format(count)

What strikes me as strange in this solution is not the use of yield itself (which can be perfectly valid) but the fact that you're losing a lot of information by turning your data into strings prematurely.

In particular, for this kind of data, simply returning a dictionary and handling the sending in the caller seems more readable:

def example():
    return {'ip': ip, 'count': count}

This also helps you separate content and presentation, which might be useful if you want, for example, to return data encoded in XML but later switch to JSON.

If you want to yield intermediate data, another possibility is using tuples: yield ('ip', ip). In this way you keep the original data and can start processing the values immediately outside the function

Upvotes: 2

Related Questions