CODEWITHSUNDEEP
pythonpython-3.xpersistent-storage
sds
sds

Reputation: 60004

How to create a variable whose value persists across file reload?

Common Lisp has defvar which creates a global variable but only sets it if it is new: if it already exists, it is not reset. This is useful when reloading a file from a long running interactive process, because it keeps the data.

I want the same in Python. I have file foo.py which contains something like this:

cache = {}
def expensive(x):
    try:
        return cache[x]
    except KeyError:
        # do a lot of work
        cache[x] = res
        return res

When I do imp.reload(foo), the value of cache is lost which I want to avoid.

How do I keep cache across reload?

PS. I guess I can follow How do I check if a variable exists? :

if 'cache' not in globals():
   cache = {}

but it does not look "Pythonic" for some reason... If it is TRT, please tell me so!

Answering comments:

Upvotes: 4

Views: 247

Answers (2)

javidcf
javidcf

Reputation: 59691

Here are a couple of options. One is to use a temporary file as persistent storage for your cache, and try to load every time you load the module:

# foo.py
import tempfile
import pathlib
import pickle

_CACHE_TEMP_FILE_NAME = '__foo_cache__.pickle'
_CACHE = {}

def expensive(x):
    try:
        return _CACHE[x]
    except KeyError:
        # do a lot of work
        _CACHE[x] = res
        _save_cache()
        return res

def _save_cache():
    tmp = pathlib.Path(tempfile.gettempdir(), _CACHE_TEMP_FILE_NAME)
    with tmp.open('wb') as f:
        pickle.dump(_CACHE, f)

def _load_cache():
    global _CACHE
    tmp = pathlib.Path(tempfile.gettempdir(), _CACHE_TEMP_FILE_NAME)
    if not tmp.is_file():
        return
    try:
        with tmp.open('rb') as f:
            _CACHE = pickle.load(f)
    except pickle.UnpicklingError:
        pass

_load_cache()

The only issue with this is that you need to trust the environment not to write anything malicious in place of the temporary file (the pickle module is not secure against erroneous or maliciously constructed data).

Another option is to use another module for the cache, one that does not get reloaded:

# foo_cache.py
Cache = {}

And then:

# foo.py
import foo_cache

def expensive(x):
    try:
        return foo_cache.Cache[x]
    except KeyError:
        # do a lot of work
        foo_cache.Cache[x] = res
        return res

Upvotes: 2

holdenweb
holdenweb

Reputation: 37003

Since the whole point of a reload is to ensure that the executed module's code is run a second time, there is essentially no way to avoid some kind of "reload detection."

The code you use appears to be the best answer from those given in the question you reference.

Upvotes: 0

Related Questions