CODEWITHSUNDEEP
pythonmultithreadingdictionarylocking
Christian
Christian

Reputation: 3393

Python: Iterating over dictionary while another thread modifies dictionary

I have a pyglet window that has a attribute "observer". The observer has a dictionary "dict". In the main_loop() function the window re-draws the window according to the content of observer.dict. The observer itself is a thread reading a stream, adding readings to dict. The observer also has a timer thread that checks each second if there are obsolete items in dict, and, if so, deletes them.

Obviously, that might cause problems if items are added or deleted while the windows iterates over dict. My current workaround is to make each time a deep copy of the dict, and draw the copy. It seems to work, but it is a ugly solution.

I'm quite new to python and particularly w.r.t Threading / Locking / etc. I guess I need the observer to "lock" the dict when adding or deleting items. Can someone give my some hints where to look first?

I've tried to extract a meaningful structrue of my code, all would be too long. I hope you can get the main idea.

class GraphConsole(window.Window):
    def __init__(self, *args, **kwargs):
        window.Window.__init__(self, *args, **kwargs)

    def init(self, observer):
        self.observer = observer


    def main_loop(self):
        while not self.has_exit:
            ...
            self.draw()

    def draw(self):
        dict_copy = deepcopy(self.observer.dict) # <-- UGLY WORKAROUND
        for k, v in dict_copy.iteritems():
           ...


class Observer(threading.Thread):
    def __init__(self):
        threading.Thread.__init__(self)
        self.dict = {}
        self.timer = Timer(1, self.delete_obsolete);
        self.timer.start()

    def run(self):
        while True:
        ...
         # read a stream
         self.dict.append(<new_element>)
         ...


    def delete_obsolete(self):
         ...
         del self.dict[...]
         ...



class Timer(threading.Thread):
    def __init__(self, interval_in_seconds, func):
         threading.Thread.__init__(self)
         self.interval_in_seconds = interval_in_seconds
         self.func = func

    def run(self):
         while True:
         self.func();
         time.sleep(self.interval_in_seconds)



if __name__ == "__main__":

    observer = Observer();
    observer.start()

    graph_console = GraphConsole()
    graph_console.init(observer)
    graph_console.main_loop()

Upvotes: 3

Views: 1577

Answers (1)

Roman Bodnarchuk
Roman Bodnarchuk

Reputation: 29707

Probably some simple lock can fix your problem. Observer class:

class Observer(threading.Thread):
    def __init__(self, lock):
        threading.Thread.__init__(self)
        self.dict_lock = lockthreading.RLock()
        self.dict = {}
        self.timer = Timer(1, self.delete_obsolete);
        self.timer.start()

    def run(self):
        while True:
        ...
         with self._dict_lock:
             # read a stream
             self.dict.append(<new_element>)
         ...


    def delete_obsolete(self):
         ...
         with self._dict_lock:
             del self.dict[...]
         ...

GraphConsole class:

class GraphConsole(window.Window):
    def __init__(self, *args, **kwargs):
        window.Window.__init__(self, *args, **kwargs)

    def init(self, observer):
        self.observer = observer

    def main_loop(self):
        while not self.has_exit:
            ...
            self.draw()

    def draw(self):
        with self.observer.dict_lock:
            for k, v in dict_copy.iteritems():
           ...

My initial answer was a bit incomplete, but I see you got the idea :)

Upvotes: 5

Related Questions