CODEWITHSUNDEEP
pythonmultithreadingpython-3.x
Matthew
Matthew

Reputation: 306

Python threading.thread.start() doesn't return control to main thread

I'm trying to a program that executes a piece of code in such a way that the user can stop its execution at any time without stopping the main program. I thought I could do this using threading.Thread, but then I ran the following code in IDLE (Python 3.3):

from threading import *
import math
def f():
    eval("math.factorial(1000000000)")
t = Thread(target = f)
t.start()

The last line doesn't return: I eventually restarted the shell. Is this a consequence of the Global Interpreter Lock, or am I doing something wrong? I didn't see anything specific to this problem in the threading documentation (http://docs.python.org/3/library/threading.html)

I tried to do the same thing using a process:

from multiprocessing import *
import math
def f():
    eval("math.factorial(1000000000)")

p = Process(target = f)
p.start()
p.is_alive()

The last line returns False, even though I ran it only a few seconds after I started the process! Based on my processor usage, I am forced to conclude that the process never started in the first place. Can somebody please explain what I am doing wrong here?

Upvotes: 5

Views: 4719

Answers (1)

jfs
jfs

Reputation: 414715

Thread.start() never returns! Could this have something to do with the C implementation of the math library?

As @eryksun pointed out in the comment: math.factorial() is implemented as a C function that doesn't release GIL so no other Python code may run until it returns.

Note: multiprocessing version should work as is: each Python process has its own GIL.

factorial(1000000000) has hundreds millions of digits. Try import time; time.sleep(10) as dummy calculation instead.

If you have issues with multithreaded code in IDLE then try the same code from the command line, to make sure that the error persists.

If p.is_alive() returns False after p.start() is already called then it might mean that there is an error in f() function e.g., MemoryError.

On my machine, p.is_alive() returns True and one of cpus is at 100% if I paste your code from the question into Python shell.

Unrelated: remove wildcard imports such as from multiprocessing import *. They may shadow other names in your code so that you can't be sure what a given name means e.g., threading could define eval function (it doesn't but it could) with a similar but different semantics that might break your code silently.

I want my program to be able to handle ridiculous inputs from the user gracefully

If you pass user input directly to eval() then the user can do anything.

Is there any way to get a process to print, say, an error message without constructing a pipe or other similar structure?

It is an ordinary Python code:

print(message) # works

The difference is that if several processes run print() then the output might be garbled. You could use a lock to synchronize print() calls.

Upvotes: 3

Related Questions