CODEWITHSUNDEEP
pythonc
user1543863
user1543863

Reputation:

Can I bypass the python GIL with a C module?

I was thinking to do some experiment with Python. I would like to introduce some feature like the JavaScript's setTimeout or setInterval, porting some sort of async behaviour without a lot of blocking calls to start an event loop.

Let's see this example (now not working of course):

from mymod import set_timeout # of course we love the PEP8 :)
import sys, time

def say_finished():
    time.sleep(2) # waiting, suppose here to have some blocking behaviour
    sys.stdout.write("\nSee you!")

sys.stdout.write("Hello, ")
set_timeout(say_finished, 2000) # supposing we have msec granularity
sys.stdout.write("World!")

I'd like to have a behaviour like JavaScript does. If set_timeout was starting a new thread because of the GIL it would still print "Hello,\nSee you!World" because the new thread blocks the main one until it finishes.

Would I be able to bypass this behaviour if set_timeout would use Posix threads in C code? It would be an integration with the Python C/APIs, and I don't know if I could bypass the GIL or if the API itself is also forced to be blocked by the GIL (I'd try but now I'm studying the docs and I'm not ready to try it out myself).

This should be then: print "Hello," -> run set_timeout (that has to return to let the interpreter to go ahead with the program execution) -> print "World!" -> stay alive while the external thread is running -> callback "say_finished" -> end.

What makes me think it would require to edit the interpreter itself is the part saying "stay alive while the external thread is running".

I don't want to use processes because of the overhead a new process creates.

Please don't say "use JavaScript then" because I love the Python syntax and I'd like to use this approach with it.

Upvotes: 2

Views: 3038

Answers (1)

snim2
snim2

Reputation: 4079

I think you're confused about the GIL. It's just an implementation detail of the VM that ensures code is thread-safe by swapping between all running threads periodically. Every VM needs some way to ensure thread-safety and while the GIL isn't necessarily the fastest or best implementation of such, it's not really something for application programmers to worry about.

You can run code in more than one thread using the threading module, or if you really want "proper" parallelism use the multiprocessing module which will run your code in separate OS processes and does "bypass the GIL", for what it's worth. If you want something specifically asynchronous, there's a new futures module that provides this sort of thing. If you actually need to sleep for a period of time just call time.sleep.

Either way, stick to something high level and, therefore, less error prone than trying to do your own application specific timing and scheduling, it's not a simple problem to solve!

Upvotes: 2

Related Questions