CODEWITHSUNDEEP
pythonpython-multiprocessing
user1561108
user1561108

Reputation: 2747

passing in an instance method into a python multiprocessing Process

If I pass in a reference to an instance method instead of a module level method into multiprocessing.Process, when I call it's start method is an exact copy of the parent process instance passed in or is the constructor called again? What happens with 'deep' instance member objects? Exact copy or default values?

Upvotes: 4

Views: 573

Answers (1)

9000
9000

Reputation: 40884

Instances are not passed between processes, because instances are per Python VM process.

Values passed between processes are pickled. Unpickling normally restores instances by measures other than calling __init__; As far as I can tell it directly sets attribute values to resurrect an instance, and resolves references to/from other instances.

Provided that you run identical code in either process (and with multiprocessing, you do), it restores a reference to a correct instance method inside a restored chain of instances.

This means that if an __init__ in the chain of objects does something side-effectful, it will not have been done on the receiving side, that is, in a subprocess. Do such initialization explicitly.

All in all, it's easiest to share (effectively) immutable objects between parallel processes, and reconcile results after join()ing all of them.

Upvotes: 3

Related Questions