CODEWITHSUNDEEP
pythonparallel-processing
Erik
Erik

Reputation: 5

Returning an Object (class) in Parallel Python

I have created a function that takes a value, does some calculations and return the different answers as an object. However when I try to parallelize the code, using pp, I get the following error.

File "trmm.py", line 8, in getattr return self.header_array[name] RuntimeError: maximum recursion depth exceeded while calling a Python object

Here is a simple version of what I am trying to do.

class DataObject(object):
    """
    Class to handle data objects with several arrays.

    """
    def __getattr__(self, name):
        try:
            return self.header_array[name]
        except KeyError:
            try:
                return self.line[name]
            except KeyError:
                raise AttributeError("%s instance has no attribute '%s'" %(self.__class__.__name__, name))

    def __setattr__(self, name, value):
        if name in ('header_array', 'line'):
            object.__setattr__(self, name, value)
        elif name in self.line:
            self.line[name] = value
        else:
            self.header_array[name] = value

class TrmmObject(DataObject):
    def __init__(self):
        DataObject.__init__(self)                            
        self.header_array = {
            'header': None
            }
        self.line = {
            'longitude': None,
            'latitude': None
            }

if __name__ == '__main__':
    import pp
    ppservers = ()
    job_server = pp.Server(2, ppservers=ppservers)
    def get_monthly_values(value):                
        tplObj = TrmmObject()
        tplObj.longitude = value
        tplObj.latitude = value * 2
        return tplObj
    job1 = job_server.submit(get_monthly_values, (5,), (DataObject,TrmmObject,),("numpy",))
    result = job1()

If I change return tplObj to return [tplObj.longitude, tplObj.latitude] there is no problem. However, as I said before this is a simple version, in reality this change would complicate the program a lot.

I am very grateful for any help.

Upvotes: 0

Views: 359

Answers (1)

Lennart Regebro
Lennart Regebro

Reputation: 172249

You almost never need to use getattr and setattr, and it almost always ends up with something blowing up, and infinite recursions is a typical effect of that. I can't really see any reason for using them here either. Be explicit and use the line and header_array dictionaries directly.

If you want a function that looks up a value over all arrays, create a function for that and call it explicitly. Calling the function __getitem__ and using [] is explicit. :-)

(And please don't call a dictionary "header_array", it's confusing).

Upvotes: 1

Related Questions