CODEWITHSUNDEEP
pythonmultiprocessing
Doug
Doug

Reputation: 118

Calling methods in Multiprocessing Manager class from another class

I'm struggling to use Python Multiprocessing Manager objects... I have this kind of structure going on:

from multiprocessing.managers import BaseManager, NamespaceProxy
from multiprocessing import RLock

class Foo(object):

    def __init__(self, a):
        self._a = a
        self._lock_a = RLock()

    @property
    def a(self):
        return self._a

    def update_a(self, new_value):
        with self._lock_a:
            self._a = new_value

    def get_bar(self):
        return Bar(self)

class FooManager(BaseManager): pass

class FooProxy(NamespaceProxy):
    _exposed_ = ('__getattribute__', '__setattr__', '__delattr__', 'a')

FooManager.register("Foo", Foo, FooProxy)


class Bar(object):

    def __init__(self, foo):
        self._foo = foo

    @property
    def a(self):
        return self._foo.a

    def update_a(self, value):
        self._foo.update_a(value)


if __name__ == "__main__":
    fmgr = FooManager()
    fmgr.start()
    foo = fmgr.Foo(5)

    bar = Bar(foo)
    print(bar.a)
    bar.update_a(10)
    print(bar.a)

It runs the first method just fine and prints the 5. However, when I go to update the value I'm getting this error:

RuntimeError: RLock objects should only be shared between processes through inheritance

Now the reason I was using manager was specifically so that calls like this, and potentially from other processes would "funnel" to the same single object. It seems like it's trying to copy/share the managed object here though. Can someone let me know if it's possible to do what I'm trying here and if so how to do this? Or do I have the wrong idea about what a Manager does?

What I want really is for the Bar object(s) to just hold the reference to the single foo manager object. In my actual implementation I have a central object which is basically a giant data manager, and then other objects which use this data and slice it up in different ways. What I want is to be able to share the super-set of data, but allow individual objects (in separate threads) to grab the data through the single manager.

Upvotes: 0

Views: 669

Answers (1)

Doug
Doug

Reputation: 118

So the work-around I found was to implement get_ functions for each property I wanted exposed, then I wrote a custom register function and AutoProxy methods that linked any properties in the class with the appropriate get_ functions. It would have been great to have been able to do this without writing explicit get_ functions, but this seemed to be the best workaround.

Upvotes: 1

Related Questions