using object inherited object inside nested class for running iteration

Question

I am building a particle filter to process data from pandas DataFrame. An idea behind the filter is to iterate to estimate best results (similar to Monte Carlo). I have a base class ExperimentalData() which has basic methods to gather the data etc. I am going to build another classes to have different filters. A filter's class is always derived from ExperimentalData().

Inside my class ParFilter(ExperimentalData) I have the method def particleFilter(self, N=1000) to run filter and get the desired estimation. As I need to have access to the data during iteration, inside I build class Iterator(object) where I going to proceed the data.

I have a problem with passing data into class Iterator(). I try most case what I thought should work but has AttributeError: 'Iterator' object has no attribute 'myData'. Eventually I am able to pass some data but it is not the same data-object I expected.

My code reduced to illustrate example:

import numpy as np
import pandas as pd

class ExperimentalData(object):
    def __init__(self):
        self.xTrueSource = 100
        self.yTrueSource = -7
        print 'source %s  %s' % (self.xTrueSource,self.yTrueSource)
        xSampPoints = np.arange(0,200)
        yTrueSignal = 100/(np.sqrt((self.xTrueSource - xSampPoints)**2 + (self.yTrueSource)**2))
        ySampPoints = yTrueSignal+np.random.randn(200)
        self.myData = pd.DataFrame({'x':xSampPoints,'ySamp':ySampPoints,'yTrue':yTrueSignal})
        #print self.myData
    def __str__(self, rows=2):           
        dfPrintStart = (self.myData[:rows]).to_string()
        dfPrintEnd =(self.myData[-rows:]).to_string()
        stringToPrint='\nPRINTNG INITIAL DATAFRAME FIRST %d ROWS and LAST %d ROWS \n %s\n...\n%s\n'\
                        % (rows, rows, dfPrintStart, dfPrintEnd)
        return stringToPrint

class ParFilter(ExperimentalData):

    def particleFilter(self, N=1000):
        '''function runs particle filter'''
        class Iterator(object):
            def __init__(self):
                '''initialise all values for iteration'''
                self.iteration = 0
                frameToWork = ParFilter().myData                
                print 'FROM CLASS Iterator.__init__ \n%s' % frameToWork
            def iterate(self):
                '''performing one step at the time'''
                self.iteration += 1
                print self.iteration
        myPartFilter = Iterator()
        for n in range(N):
            myPartFilter.iterate()
        return myPartFilter

if __name__ == '__main__':
    data = ParFilter()
    print data
    data.particleFilter(10)

The problem is that when I initialise my class I have dataFrame with particular values, but when I do the step: frameToWork = ParFilter().myData instead of taking the same data-object I produce new object with different data. Snapshot of the output:

PRINTNG INITIAL DATAFRAME FIRST 2 ROWS and LAST 2 ROWS 
   x     ySamp     yTrue
0  0  0.510414  0.997559
1  1  1.522934  1.007585
...
       x     ySamp     yTrue
198  198  1.508216  1.017815
199  199  2.409181  1.007585

FROM CLASS Iterator.__init__ 
   x     ySamp     yTrue
0  0  0.727060  0.997559
1  1  0.631976  1.007585

The first value of ySamp in initialisation is 0.510414 and it should be the same in Iterator instead of 0.727060. So I create new object.

I cannot figure out how to get the original myData object into Iterator I try:

    class Iterator(ParFilter):
        def __init__(self):
            '''initialise all values for iteration'''
            self.iteration = 0
            frameToWork = self.myData 

with AttributeError: 'Iterator' object has no attribute 'myData'.

I try: class Iterator(self.ParFilter) with AttributeError: 'ParFilter' object has no attribute 'ParFilter' and some more but no results.

(I have to use pandas DataFrame as my base class is quite large and got large dataFrame not like in example)

Answer 1

The problem your code is that the inner class is trying to access a member variable of the outer class. This is impossible since they're both using self to refer to their current instance, and the inner class's self parameter shadows the outer class's self. You need to use a different name for one of them.

While you could actually use a different name for the first parameter name in one of the methods, I'd suggest simply binding an additional name to the outer self object before defining the nested class:

class Outer(object):
    def __init__(self):
        self.foo = "foo"

    def do_stuff(self):
        outer_self = self # give an extra name to `self` that won't be shadowed

        class Inner(object):
            def __init__(self):
                 self.bar = "bar"

            def do_inner_stuff(self):
                print(outer_self.foo, self.bar) # access the outer class's data

        i = Inner()
        i.do_inner_stuff()

This works, but it's still probably not the best design. Nested classes are unserializable, can be pretty nasty to debug and should probably be avoided if possible.

A better idea is to unnest your classes and simply pass the data you need from the outer class to the inner class's constructor where a reference can be saved as a member variable:

class Outer(object):
    def __init__(self):
        self.foo = "foo"

    def do_stuff(self):
        i = Inner(self.foo) # pass relevant data to constructor
        i.do_inner_stuff()

class Inner(object):
    def __init__(self, foo):
        self.foo = foo # keep a reference to passed data
        self.bar = "bar"

    def do_inner_stuff(self):
        print(self.foo, self.bar)  # use the data