CODEWITHSUNDEEP
pythonmultiprocessingpython-3.4
Jonathan
Jonathan

Reputation: 2845

Where is multiprocessing.Process

I am porting a working application from Python 3.3 to 3.4 and have encountered a weird situation. The class multiprocessing.Process is not present in the download from python.org. Instead, in the multiprocessing.process module, I find a class multiprocessing.process.BaseProcess. The only trace of the old Process class that I can find is in the new multiprocessing.context module where multiprocessing.context.Process is basically a cover function for BaseProcess. None of this is mentioned in the documentation for Python 3.4. Can anyone tell me what is going on and possibly point me at some documentation.

Upvotes: 1

Views: 1268

Answers (2)

frmdstryr
frmdstryr

Reputation: 21362

If you're using Eclipse and PyDev you need to include 'multiprocessing' as a forced builtin for the python interpreter.

Upvotes: 3

dano
dano

Reputation: 94901

It's still there, it's just defined in a very different way:

multiprocessing/__init__.py now adds attributes to the top-level package like this:

#
# Copy stuff from default context
#

globals().update((name, getattr(context._default_context, name))
                 for name in context._default_context.__all__)
__all__ = context._default_context.__all__

So it's pulling __all__ from context._default_context.

If you look in context.py, you can find a Process class:

#
# Type of default context -- underlying context can be set at most once
#

class Process(process.BaseProcess):
    _start_method = None
    @staticmethod
    def _Popen(process_obj):
        return _default_context.get_context().Process._Popen(process_obj)

And more concrete implementations are referenced and defined in there, too:

class ForkProcess(process.BaseProcess):
    _start_method = 'fork'
    @staticmethod
    def _Popen(process_obj):
        from .popen_fork import Popen
        return Popen(process_obj)

class SpawnProcess(process.BaseProcess):
    _start_method = 'spawn'
    @staticmethod
    def _Popen(process_obj):
        from .popen_spawn_posix import Popen
        return Popen(process_obj)

class ForkServerProcess(process.BaseProcess):
    _start_method = 'forkserver'
    @staticmethod
    def _Popen(process_obj):
        from .popen_forkserver import Popen
        return Popen(process_obj)

class ForkContext(BaseContext):
    _name = 'fork'
    Process = ForkProcess

class SpawnContext(BaseContext):
    _name = 'spawn'
    Process = SpawnProcess

class ForkServerContext(BaseContext):
    _name = 'forkserver'
    Process = ForkServerProcess
    def _check_available(self):
        from . import reduction
        if not reduction.HAVE_SEND_HANDLE:
            raise ValueError('forkserver start method not available')

_concrete_contexts = {
    'fork': ForkContext(),
    'spawn': SpawnContext(),
    'forkserver': ForkServerContext(),
}

_default_context = DefaultContext(_concrete_contexts['fork'])

the __all__ for default context gets set a little later:

DefaultContext.__all__ = list(x for x in dir(DefaultContext) if x[0] != '_')

The changes in the code's layout were made to support contexts. For most people this change should have been transparent, but anyone who was importing classes from outside of the top-level package (e.g. from multiprocessing.queues import Queue or from multiprocessing.process import Process) will see things break on them.

Upvotes: 1

Related Questions