CODEWITHSUNDEEP
pythontwistedblockinginotify
Simon
Simon

Reputation: 115

python twisted INotify without blocking reactor

I am using twsited's INotify to monitor the /dev directory to monitor for new serial devices being added. The code I am currently using is similar to below.

notifier = INotify()
notifier.watch(FilePath("/dev"), IN_CREATE, callbacks=[self.created])
notifier.startReading()

def created(self, ignored, path, mask):
    ...
    blocking code
    ...

The problem I am having at the moment is when 'created' is getting called, it is blocking my reactor so other network sessions (I have both TCP and UDP connections associated with the same reactor) have to wait for the 'created' method to finish.

Does anyone know how I can make the 'created' method run in the background so it doesn't block my reactor?

Thanks,

Simon

Upvotes: 1

Views: 1820

Answers (2)

Jean-Paul Calderone
Jean-Paul Calderone

Reputation: 48345

All event handlers in Twisted run in the "reactor thread" - UDP, TCP, and indeed inotify. They all are expected to cooperate with the system by not blocking. So, in this sense, this is just a question about how to write good event handlers in Twisted, not about inotify in particular.

There are lots of options for avoiding blocking. The tricky thing about answering your question is that the right option depends on why exactly the current code blocks.

Does it do socket I/O? Use Twisted's non-blocking socket I/O APIs instead.

Does it do filesystem I/O? You may need to use a thread here, since non-blocking filesystem I/O is difficult (perhaps not impossible) without one.

Does it talk to a SQL database? Perhaps twisted.enterprise.adbapi can help.

And so on.

I don't know if that covers the case you're in. However, I'll emphasize two things. First, it is perfectly reasonable to use threads in a Twisted program. Much of Twisted exists so you won't have to use threads, but if you come to a circumstance where threads get the job done and nothing else does - go for it (with caution ;). Twisted even has some helpers to make it easier, such as the deferToThread mentioned by zeekay. Second, pick the appropriate solution for the task. The collection of all "blocking" problems is only slightly smaller than the collection of all general programming problems. There are lots of possible solutions. Some, like threads, seem to have a wide range of applicability, but with a little care you might find something more specifically suitable to a particular circumstance.

Additionally, take a look at Twisted: Making code non-blocking for some further explanation.

Upvotes: 7

Zach Kelling
Zach Kelling

Reputation: 53879

You could use twisted.internet.threads.deferToThread to run your blocking code in a thread:

deferToThread(self.created, ignored, path mask)

Upvotes: 1

Related Questions