Writing multiple variables to a file using a function

Question

I am writing a function that exports variables as a dictionary to an external file. The problem comes when calling that function from another script. I think it has something to do with the globals() parameter.

import sys
import os

mydict = {}  #'initialising" the an empty dictionary to be used locally in the function below

def writeToValues(name):
    fileName = os.path.splitext(os.path.basename(sys.argv[0]))[0]
    valuePrint=open("values.py","a")
    def namestr(obj,namespace):
        return[name for name in namespace if namespace[name] is obj]
    b = namestr(name, globals())
    c = "".join(str(x) for x in b)
    mydict[(c)] = name
    valuePrint.write(fileName)
    valuePrint.write("=")
    valuePrint.write(str(mydict))
    valuePrint.write("
")
    valuePrint.close()
    return mydict

a = 2
b = 3

writeToValues(a)
writeToValues(b)

I get the following result:

Main Junkfile={'a': 2, 'b': 3}

note the word Main Junkfile is the name of the script I ran as that is what the function first does, to get the name of the file and use that to name the dictionary.

Now help me as I cannot generate the same if I import the function from another script. Another problem is that running the script twice generates the values in steps.

Main Junkfile={'a': 2}
Main Junkfile={'b': 3, 'a': 2}

I cannot change the file open mode from append to write since I want to store values from other scripts, too.

zwer · Accepted Answer

First of all, to get the current executing script name, or rather the module that called your function you'll have to pick it up from the stack trace. Same goes for globals() - it will execute in the same context of writeToValues() function so it won't be picking up globals() from the 'caller'. To remedy that you can use the inspect module:

import inspect
import os

def writeToValues(name):
    caller = inspect.getmodule(inspect.stack()[1][0])
    caller_globals = caller.__dict__  # use this instead of globals()
    fileName = os.path.splitext(os.path.basename(caller.__file__))[0]
    # etc.

This will ensure that you get the name of the module that imported your script and is calling writeToValues() within it.

Keep in mind that this is a very bad idea if you intend to write usable Python files - if your script name has spaces (like in your example) it will write a variable name with spaces, which will further result in syntax error if you try to load the resulting file into a Python interpreter.

Second, why in the name of all things fluffy are you trying to do a reverse lookup to find a variable name? You are aware that:

a = 2
b = 2
ab = 5

writeToValues(b)

will write {"ab": 2}, and not {"b": 2} making it both incorrect in intent (saves the wrong var) as well as in state representation (saves a wrong value), right? You should pass a variable name you want to store/update instead to ensure you're picking up the right property.

The update part is more problematic - you need to update your file, not just merely append to it. That means that you need to find the line of your current script, remove it and then write a new dict with the same name in its place. If you don't expect your file to grow to huge proportions (i.e. you're comfortable having it partially in the working memory), you could do that with:

import os
import inspect

def writeToValues(name):
    caller = inspect.getmodule(inspect.stack()[1][0])
    caller_globals = caller.__dict__  # use this instead of globals()
    caller_name = os.path.splitext(os.path.basename(caller.__file__))[0]
    # keep 'mydict' list in the caller space so multiple callers can use this
    target_dict = caller_globals['mydict'] = caller_globals.get('mydict', {})
    if name not in caller_globals:  # the updated value no longer exists, remove it
        target_dict.pop(name, None)
    else:
        target_dict[name] = caller_globals[name]
    # update the 'values.py':
    # optionaly check if you should update - if values didn't change no need for slow I/O
    with open("values.py", "a+") as f:
        last_pos = 0  # keep the last non-update position
        while True:
            line = f.readline()  # we need to use readline() for tell() accuracy
            if not line or line.startswith(caller_name):  # break at the matching line or EOF
                break
            last_pos = f.tell()  # new non-update position
        append_data = f.readlines()  # store in memory the rest of the file content, if any
        f.seek(last_pos)  # rewind to the last non-update position
        f.truncate()  # truncate the rest of the file
        f.write("".join((caller_name, " = ", str(target_dict), "
")))  # write updated dict
        if append_data:  # write back the rest of the file, if truncated
            f.writelines(append_data)
    return target_dict

Otherwise use a temp file to write everything as you read it, except for the line matching your current script, append the new value for the current script, delete the original and rename the temp file to values.py.

So now if you store the above in, say, value_writter.py and use it in your script my_script.py as:

import value_writter

a = 2
b = 3

value_writter.write_to_values("a")
value_writter.write_to_values("b")

a = 5

value_writter.write_to_values("a")

# values.py contains: my_script = {"a": 5, "b": 3}

Same should go for any script you import it to. Now, having multiple scripts edit the same file without a locking mechanism is an accident waiting to happen, but that's a whole other story.

Also, if your values are complex the system will break (or rather printout of your dict will not look properly). Do yourself a favor and use some proper serialization, even the horrible pickle is better than this.

Writing multiple variables to a file using a function

Answers (2)

Related Questions