iteratively replace values in json file

Question

I have to perform some tests to tune some numeric parameters of a json file. To make simple, I replaced all these values by the string 'variable', then did the following:

numeric_vals = [10,20, 30, 40]  # numeric values to replace in that order 
with open ('mypath') as my_file:
    json_str = my_file.read()
for i in numeric_vals:
    json_str = json_str.replace("\"variable\"", str(i), 1)
c = json.loads(json_str)  #loading in order to work with

This works fine, but is there a more efficient way to do this ? Values that need to be replaced are in variable depths, and may be inside lists etc. My json file is 15Kb and I need to test many (really many!) configurations. At each test, about 200 variables need to be replaced. I am on python 2.7 , but python 3.5 is also an option. Thanks for your help!

EDIT :

here is a sample of my dict. It should be noted that the real thing is far longer and deeper:

 {
"1": {
    "transition": {
        "value": "variable", # edit here
        "unite": "sec"
    },
    "sequence": [
        {
            "step": "STEP",
            "name": "abc",
            "transition": {
                "value": "variable", #edit here
                "unite": "sec"
            },
            "entity": "aa",
            "_equipement": 3,
            "_value": 0
        },
        {
            "step": "FORK",
            "BRANCHES": [
                {
                    "": {
                        "sequence": [
                            {
                                "step": "STEP",
                                "name": "klm",
                                "transition": {
                                    "value": "variable", # edit here
                                    "unite": "sec"
                                },
                                "entity": "vvv",
                                "_equipement": 10,
                                "_value": 0,
                                "conditions": [
                                    [
                                        {
                                            "name": "ddd",
                                            "type": "el",
                                            "_equipement": 7,
                                            "_value": 1
                                        }
                                    ]
                                ]
                            }
                        ]
                    }
                },
                {
                    "SP": {
                        "sequence": [
                            {
                                "step": "STEP",
                                "name": "bbb",
                                "transition": {
                                    "value": "variable", # edit here
                                    "unite": "sec"
                                },
                                "entity": "bb",
                                "_equipement": 123,
                                "_value": 0,
                                "conditions": [
                                    [
                                        {
                                            "name": "abcd",
                                            "entity": "dgf",
                                            "type": "lop",
                                            "_equipement": 7,
                                            "_valeur": 0
                                        }
                                    ]
                                ]
                            }
                        ]
                    }
                }
            ]
        }
    ]
}

}

Answer 1

It's generally a bad idea to do string operations on hierarchical/structured data as there may be many cases where you can break the structure. Since you're already parsing your JSON you can extend the decoder to specifically deal with your case during parsing, e.g.:

numeric_vals = [10, 20, 30, 40]  # numeric values to replace in that order

SEARCH = 'variable'
REPLACE = iter(numeric_vals)  # turn into an iterator for sequential access

def replace_values(value):
    return {k: next(REPLACE) if v == SEARCH else v for k, v in value.items()}

with open('path/to/your.json') as f:
    a = json.JSONDecoder(object_hook=replace_values).decode(f.read())

This ensures you're properly parsing your JSON and that it won't replace, for example, a key that happens to be called 'variable'.

Beware, tho, that it will raise an StopIteration exception if there are more "variable" values in the JSON than there are numeric_vals - you can unravel the dict comprehension in replace_values and deal with such case if you expect to encounter such occurrences.

Answer 2

You can get a performance improvement by taking the call to json.loads() outside of the loop, you only need to do this once at the end:

numeric_vals = [10, 20, 30, 40]

with open('mypath') as my_file:
    json_str = my_file.read()
    for i in numeric_vals:
        json_str = json_str.replace('"variable"', str(i), 1)
    c = json.loads(json_str)

Also prefer to use string.replace over re.sub, as per this post.