CODEWITHSUNDEEP
pythonstatic
Vitaly Trifanov
Vitaly Trifanov

Reputation: 631

Initialize field only once in Python

I have file services.py with certain class MyCache in it. All instances of MyCache should share one "cache" field, so I made it static. In order to initialize cache there is static method that loads it. This method is called exactly once at the very start of the app.

The problem is that when I import services.py from other .py file and create instance of MyCache - it prints that cache is empty!

How can I fix it and make "cache" field shared by all instances of the class disregard of place from which they are created?

I can't get why this happens. Please help :)

services.py:

class MyCache:
    cache = {}

    @staticmethod
    def initialize():
       MyCache.cache = load_from_file()
       print(len(MyCache.cache)) # prints 3

    def __init__(self):
       print(len(MyCache.cache)) # supposed to print 3, because initialize has been called earlier, but prints 0 when called from other_file.py

main.py:

import services

if __name__ == '__main__':
    services.MyCache.initialize()

other_file.py:

import services

services.MyCache().foo() # creates instance, prints 0 in MyCache.__init__

Upvotes: 10

Views: 4403

Answers (4)

wwii
wwii

Reputation: 23773

This might work - add the class attribute if it doesn't already exist using a metaclass:

foo.py:

def load_stuff():
    return {'foo':1, 'bar':2}

class F(type):
    def __new__(meta, name, bases, namespace):
        if 'cache' not in namespace:
            print('adding cache')
            namespace['cache'] = load_stuff()
        return super().__new__(meta, name, bases, namespace)

class MyCache(metaclass = F):
    def __init__(self):
       print(len(MyCache.cache))

test.py:

print(__name__)
import foo
print(foo.MyCache.cache)
print('********************')

tmp.py:

print('tmp.py')
import foo
print('*******************')
import test

>>> import tmp
tmp.py
adding cache
*******************
test
{'foo': 1, 'bar': 2}
********************
>>> tmp.foo.MyCache.cache
{'foo': 1, 'bar': 2}
>>> tmp.test.foo.MyCache.cache
{'foo': 1, 'bar': 2}
>>> tmp.test.foo.MyCache.cache['x'] = 'x'
>>> tmp.test.foo.MyCache.cache
{'foo': 1, 'bar': 2, 'x': 'x'}
>>> tmp.foo.MyCache.cache
{'foo': 1, 'bar': 2, 'x': 'x'}
>>> 
>>> tmp.foo.MyCache.cache is tmp.test.foo.MyCache.cache
True
>>>

>>> import test
test
adding cache
{'foo': 1, 'bar': 2}
********************
>>> test.foo.MyCache.cache
{'foo': 1, 'bar': 2}
>>> 
>>> import tmp
tmp.py
*******************
>>>
>>> tmp.foo.MyCache.cache
{'foo': 1, 'bar': 2}
>>>
>>> tmp.foo.MyCache.cache['x'] = 'x'
>>> tmp.foo.MyCache.cache
{'foo': 1, 'bar': 2, 'x': 'x'}
>>> test.foo.MyCache.cache
{'foo': 1, 'bar': 2, 'x': 'x'}
>>>
>>> z = tmp.foo.MyCache()
3
>>> z.cache
{'foo': 1, 'bar': 2, 'x': 'x'}
>>>
>>> z.cache['y'] = 'y'
>>> z.cache
{'foo': 1, 'bar': 2, 'x': 'x', 'y': 'y'}
>>> test.foo.MyCache.cache
{'foo': 1, 'bar': 2, 'x': 'x', 'y': 'y'}
>>> tmp.foo.MyCache.cache
{'foo': 1, 'bar': 2, 'x': 'x', 'y': 'y'}
>>>
>>> tmp.foo.MyCache.cache is test.foo.MyCache.cache
True

I started thinking and realized that the class attribute could also be a singleton that inherits from dict.

temp.py and test.py - same as above

foo.py:

def load_stuff():
    return [('a', 1), ('b', 2)]

class Borg:
    _shared_state = {}
    def __new__(cls, *a, **k):
        obj = super().__new__(cls, *a, **k)
        obj.__dict__ = cls._shared_state
        return obj

class Cache(dict, Borg):
    pass

class OneCache(metaclass = F):
    cache = Cache(load_stuff())
    def __init__(self):
       print(len(OneCache.cache))

Then:

>>> import tmp
>>> tmp.foo.OneCache.cache
{'a': 1, 'b': 2}
>>> tmp.test.foo.OneCache.cache
{'a': 1, 'b': 2}
>>> z = tmp.foo.OneCache()
2
>>> z.cache['r'] = 't'
>>> z.cache
{'a': 1, 'b': 2, 'r': 't'}
>>> tmp.foo.OneCache.cache
{'a': 1, 'b': 2, 'r': 't'}
>>> tmp.test.foo.OneCache.cache
{'a': 1, 'b': 2, 'r': 't'}
>>> 
>>> tmp.foo.OneCache.cache is tmp.test.foo.OneCache.cache is z.cache
True
>>>

Upvotes: 0

tdelaney
tdelaney

Reputation: 77387

One problem is that you have modules using the class during import before execution has reached the if __name__ == '__main__: part that does the initialization.

You can use a classmethod to initialize the class-level cache dynamically on first use. Add a lock and it is also thread-safe. You no longer need to initialize specifically in __main__, which is easy to forget, and you can use it at any time by other importers.

import threading

class MyCache:
    cache = None
    _lock = threading.Lock()

    @classmethod
    def initialize(cls):
       with cls._lock:
           if cls.cache is None:
               cls.cache = load_from_file()

    def __init__(self):
       self.initialize()       
       print(len(MyCache.cache))

Upvotes: 1

user993533
user993533

Reputation: 81

    class MyCache:
        cache = {}
        __initialized = False

        @staticmethod
        def initialize():
           if not MyCache.__initialized:
               MyCache.cache = load_from_file()
               MyCache.__initialized = True

        def __init__(self):
           print(len(MyCache.cache))

Upvotes: 0

gnicholas
gnicholas

Reputation: 2077

#mycache.py
def load_from_file():
    pass
    ...
cache = load_from_file()

#anotherlib.py
from mycache import cache

...

#main.py
from anotherlib import ... #(Now the cache is initialized)
from mycache import cache #(Python looksup the mycache module and doesn't initialize it again)

Here we are just using a python module as a singleton. To learn more about how python caches modules so they are only initialized once, read here: https://docs.python.org/2/library/sys.html#sys.modules

Upvotes: 1

Related Questions