CODEWITHSUNDEEP
pythonmultithreadingmultiprocessing
VictorVH
VictorVH

Reputation: 327

Python multiprocessing not working

I've read that you can use multiprocessing and multithreading the same way. I've tried doing this with multithreading but it is not working with multiprocessing.

I'm running the following code:

import multiprocessing
import time


test_list = []


def user_input():
    while True:
        number = raw_input('Please input a number: ')
        test_list.append(number)


def print_func():
    while True:
        for t in test_list:
            print t

        time.sleep(5)


if __name__ == '__main__':
    ui = multiprocessing.Process(target=user_input)
    p = multiprocessing.Process(target=print_func)

    ui.start()
    p.start()

I am getting the following error for this program:

Traceback (most recent call last):
  File "C:\Python27\lib\multiprocessing\process.py", line 258, in _bootstrap
    self.run()
  File "C:\Python27\lib\multiprocessing\process.py", line 114, in run
    self._target(*self._args, **self._kwargs)
  File "C:\Users\Victor\Dropbox\Private\multiFile\main.py", line 10, in user_input
    number = raw_input('Please input a number: ')
EOFError: EOF when reading a line

Upvotes: 0

Views: 465

Answers (1)

abarnert
abarnert

Reputation: 365617

I've read that you can use multiprocessing and multithreading the same way.

Up to a certain point, yes. But you're going beyond that point.

The first problem is that, by default, child processes do not get stdio. This is explained in Programming guidelines:

multiprocessing originally unconditionally called:

os.close(sys.stdin.fileno())

in the multiprocessing.Process._bootstrap() method — this resulted in issues with processes-in-processes. This has been changed to:

sys.stdin.close()
sys.stdin = open(os.devnull)

Part of the reason for this is that it's dangerous and difficult to share stdio pipes across multiple processes like this. What you want to do is detach stdin in the parent process, then explicitly pass the original stdin to whichever child you want to take over.

The second problem is that you're trying to share a global variable, test_list, between processes.

As explained in the section on programming guidelines for Windows:

Global variables

Bear in mind that if code run in a child process tries to access a global variable, then the value it sees (if any) may not be the same as the value in the parent process at the time that Process.start was called.

In other words, each child process will have its own copy of test_list, completely independent from each other. Changes you make in the user_input process will not be visible in the print_func process.

On top of that, you're code isn't really safe even with threads. If you share mutable objects between threads (or processes), you need to lock them. (Yes, in many cases, when using built-in types, on CPython, the GIL happens to make things safe for you, but you can either learn exactly when it's safe to get away with it and how to debug things when you get it wrong, or you can just learn how to lock things properly.)

So, how do you share state between processes? Well, there's a whole section called Sharing state between processes in the docs that explains it. But it's generally not easy or fun. Which is why that section starts off with:

As mentioned above, when doing concurrent programming it is usually best to avoid using shared state as far as possible. This is particularly true when using multiple processes.

A better solution is to pass messages around instead of sharing state. This is descriebd a couple sections up in Exchanging objects between processes, and it's a whole lot simpler for most use cases.

Upvotes: 3

Related Questions