twisted - queue a function interactively

Question

I've just been introduced to twisted through this nice tutorial for a chatting app, but I'm unsure how to adapt it to my needs.

Suppose that, on a whim, I wanted the chat server to send a friendly message out to all clients, like "happy long weekend!". i.e. I want to get the reactor to run something, but after it is already running (so I can't schedule it in advance, or I don't want to).

I want to do something like this:

def do_something():
    # do something

# setup and run reactor
factory = Factory()
factory.clients = []
factory.protocol = MyServer 
reactor.listenTCP(80, factory)
reactor.run() # asynchronously?

# clients connect...

reactor.callLater(0, do_something)

I tried using python threading but it didn't work. I looked at these twisted examples but they all have the reactor.run() statement last, which throws me off. Yes, it's quite likely I'm missing something fundamental (which is why I'm here).

Answer 1

You've said:

Suppose that, on a whim, I wanted ... to send a friendly message out to all clients

and

I guess what I want to do then is generate events interactively in Python, when I'm sitting at the server.

I'm going to translate that to "I want to have a keyboard interface when my reactor is running" and give you an example of that.

In Twisted, the keyboard is just another IO interface you can work with along with all other IO, the example I'm going to provide is for unix/posix type platforms though the same idea can certainly be implemented on other OSs too.

(Disclamer: this example is a bit messy because it is setting cbreak mode on the tty, this is something I like to do for interactive control but its certainly not required.)

#!/usr/bin/python

import sys # so I can get at stdin
import os # for isatty
import termios, tty # access to posix IO settings
from twisted.internet import reactor
from twisted.internet import stdio # the stdio equiv of listenXXX
from twisted.protocols import basic # for lineReceiver for keyboard
from twisted.internet.protocol import Protocol, ServerFactory

class Cbreaktty(object):
    org_termio = None
    my_termio = None

    def __init__(self, ttyfd):
        if(os.isatty(ttyfd)):
            self.org_termio = (ttyfd, termios.tcgetattr(ttyfd))
            tty.setcbreak(ttyfd)
            print '  Set cbreak mode'
            self.my_termio = (ttyfd, termios.tcgetattr(ttyfd))
        else:
          raise IOError #Not something I can set cbreak on!

    def retToOrgState(self):
        (tty, org) = self.org_termio
        print '  Restoring terminal settings'
        termios.tcsetattr(tty, termios.TCSANOW, org)


class MyClientConnections(Protocol):
    def connectionMade(self):
        print "Got new client!"
        self.factory.clients.append(self)

    def connectionLost(self, reason):
        print "Lost a client!"
        self.factory.clients.remove(self)

class MyServerFactory(ServerFactory):
    protocol = MyClientConnections

    def __init__(self):
        self.clients = []

    def sendToAll(self, message):
      for c in self.clients:
        c.transport.write(message)

    def hello_to_all(self):
        self.sendToAll("A friendly message, sent on a whim\n")
        print "sending friendly..."

class KeyEater(basic.LineReceiver):

    def __init__(self, hello_callback):
        self.setRawMode() # Switch from line mode to "however much I got" mode
        self.hello_to_all = hello_callback

    def rawDataReceived(self, data):
        key = str(data).lower()[0]
        if key == 's':
            self.hello_to_all()
        elif key == 'q':
            reactor.stop()
        else:
            print "Press 's' to send a message to all clients, 'q' to shutdown"

def main():
    client_connection_factory = MyServerFactory()

    try:
      termstate = Cbreaktty(sys.stdin.fileno())
    except IOError:
      sys.stderr.write("Error: " + sys.argv[0] + " only for use on interactive ttys\n")
      sys.exit(1)

    keyboardobj = KeyEater(client_connection_factory.hello_to_all)

    stdio.StandardIO(keyboardobj,sys.stdin.fileno())
    reactor.listenTCP(5000, client_connection_factory)
    reactor.run()
    termstate.retToOrgState()
if __name__ == '__main__':
  main()

If you run the above code (.. assuming your on unix/posix) you will have a reactor that is both waiting-for/servicing TCP connections and waiting for keys to happen on stdin. Typing the key 's' will send a message to all connected clients.

This is the method I regularly use for async control of twisted apps, though its certainly only one of many as noted in the other answers.

Answer 2

As JP says, the reactor responds to an event. Assuming that you're running on a Linux/Unix system:

• You could start a thread from within a Twisted factory or protocol and have the thread send a command back using reactor.callInThread() and reactor.callFromThread()
• Or you could set a callLater from a function initiated by callWhenRunning to run a function that checks for a file or directory or whatever and performs some action if it's found. The function run by the callLater could potentially set another callLater for subsequent action.
• or you could set up a looping call from within a Twisted factory ( or even from within a Protocol)
• or, your chat server could watch for a special sequence of characters from a client that instruct the server to do something (like send an attached message to all logged in clients
• or your server can (from within twisted), monitor a ip or unix 'command' socket watching for commands that cause a message to be sent.
• In your context, an event could be an action initiated by a Unix signal handler, although as I recall Twisted does something internally to intercept signals so some research would be required.

I've struggled to obtain an asynchronous Twisted mindset for about 6 months now and I'm finally starting to 'get it'. It takes a while but it's a worthwhile journey.

To some people, learning twisted is like one of those 80's vintage text based adventure games where you more or less stumble around experimentally until you discover a magic sack that attracts helpful devices. At that point things get easier.

I've found that studying the twisted source code is very informative, but there's a lot of it and knowing where to start or find things can be challenging. The code tends to be well organized and clear and more or less consistent, which helps.

The other positive is that people who are very involved with Twisted (Glyph and JP are just 2 of many) are available here to help.

If time ever allows I plan to put together a group of example servers and clients that exercise more of the bells and whistles of twisted than the examples that are currently available do. I'd hope that after doing this I could get them reviewed by the Twisted folk who might then consider making them available to others.

Good luck with your Twisted journey

Answer 3

You don't need to change the way you run the reactor to implement this behavior.

Instead, just recognize that everything in your program is a response to some event.

When do you send out a "happy long weekend" notification? When a long weekend is about to start, of course. In other words, a calendar (which is just a special kind of time keeping device) generates an event and you react to it. You can implement this using IReactorTime.callLater: compute the time until the next long weekend and reactor.callLater(that_delay, some_function).

If you want to do something when a user clicks a button, that's response to an event generated by a GUI library. If you want to do something when a USB device is connected, that's a response to an event generated by the platform HAL (or something like DBUS or udev).

Any time you think you "act on its own" think about why it is acting - under what condition or in response to what circumstances - and you'll be on your way to figuring out what event it is actually reacting to.