CODEWITHSUNDEEP
pythonasynchronoustwisted
ducin
ducin

Reputation: 26467

understanding python twisted asynchronicity in terms of operating system

I'm new to the twisted library and I'm trying to understand how it is done that operations in python/twisted are performed asynchronously. So far I thought that only GUI-alike (Qt or javascript) platforms use event-driven architecture extensively.

facts:

question 1

Python could be seen as a high-level wrapper for the operating system. What are the OS functions (or C functions) that provide asynchronous operation handling? Are there any?

question 2

Q1 leads me to an idea, that twisted's asynchronicity is not a true asynchronicity, like we have in Javascript. In JavaScript, for example, if we provide 3 different buttons, attach callback functions to them and we click all three buttons - then the 3 callbacks will be executed parallelly. Truly parallelly.

In Twisted - as far as I understand - it's not true asynchronicity - it's, let's say, approximated asynchronicity instead, since no operations would be performed parallelly (in terms of code, as I mentioned in fact3). In Twisted the first n line of code (defining protocols, factories, connections, etc.) are the declarations of what is going to happen when entire system starts. Nothing runs so far. Real execution starts then the reactor.run() is fired. I understand that the reactor runtime is based on a single while True loop which iterates through events. The reactor checks any awaiting tasks to do, processes them, send their result back to the queue (either to callbacks or errbacks). In the next loop execution they'll be processed one step further. So the deferred execution is linear in fact (though, from outside it looks like it was executed parallelly). Is my interpretation correct?

I'd appreciate if someone could answer my questions and/or explain how asynchronicity works in twisted/python platform and how is it related to operating system. Thanks in advance for good explanations!

edit: links to articles explaining asynchronicity are very welcome!

Upvotes: 6

Views: 413

Answers (3)

Glyph
Glyph

Reputation: 31880

I gave a half-hour talk on this exact topic at PyCon 2012. You can watch it online here.

Upvotes: 2

Avatar33
Avatar33

Reputation: 816

Thomas has already answered your first question but I'll like to add something extra to question 2. From the way you phrased the (second) question, it seems you many misunderstand asynchronicity. Asynchronous should not be confused with multiprocessing. Here's an example of asynchronicity. Lets say you have two tasks to complete. 1) Read a file from disk 2) Sum a couple of integers in memory

In a synchronous system, these task are done one at a time and we wait for the result of the operation before moving on to the next. In an asynchronous system we start each operation and then periodically check for the completion of each. (This is where that nifty select operation comes in) See this wiki page for more info: http://en.wikipedia.org/wiki/Asynchronous_I/O

So even on a single core system where in reality only 1 thread is running at a particular point in time, we can still have an asynchronous system so that tasks that take long (reading a file in the above example) don't throw a spanner in the works for tasks that can complete their work quickly (summing some integers in memory)

(As a side note, Twisted does have support for spawning a new threads. http://twistedmatrix.com/documents/11.0.0/core/howto/threading.html )

Upvotes: 2

Thomas
Thomas

Reputation: 6762

It's hard to talk about this without defining a lot of terms more precisely and taking issue with your facts, but here's my attempt:

Question 1:

Try man select, which is approximately how Twisted is implemented - it's a way to ask the operating system to monitor several things at once and let the application know when any one of them fires (block on multiple things).

Question 2:

Yeah, pretty much - but you're wrong about Javascript, it's just like Twisted.

Upvotes: 5

Related Questions