how to modify the key of a nested Json

Question

I am trying to update the keys of a JSON object which looks like this:

results = 
 {  
   'Game':12345,
   'stats':[  
      {  
         'detail':[  
            {  
               'goals':4,
               'refs':{  
                  'number':0

I am currently manually updating each key as follow

##update Game to newValue
results['newValue'] = results['Game']
    del results['Game']

## update nested key "goals" to "goals_against"
results['stats'][0]['detail'][0]['goals_against'] = results['stats'][0]['detail'][0]['goals']
    del results['stats'][0]['detail'][0]['goals']

there has to be a better way to do as I am finding myself having to update multiple keys on results. For example, I also want to update the "number" key to "assis_ref".

I know how to update a key if is the json file is "simple": ie if i could do this:

result['stats']['details']['refs']

however, 'stats' and 'details' require [0] next to it which i assume is the index of the element i am trying to go next.

Answer 1

I wrote a recursive function to handle transforming keys in json objects. Especially useful for deeply nested json!

def walk_json(obj, key_transform):
    """
    Recurse over a json object modifying the keys according to the `key_transform` function.
    Returns a new json object with the modified keys.
    """
    assert isinstance(obj, dict), "walk_json expects obj to be of type dict"

    def _walk_json(obj, new):

        if isinstance(obj, dict):

            if isinstance(new, dict):
                for key, value in obj.items():

                    new_key = key_transform(key)

                    if isinstance(value, dict):
                        new[new_key] = {}
                        _walk_json(value, new=new[new_key])

                    elif isinstance(value, list):
                        new[new_key] = []
                        for item in value:
                            _walk_json(item, new=new[new_key])

                    else:  # take value as is
                        new[new_key] = value

            elif isinstance(new, list):
                new.append(_walk_json(obj, new={}))

        else:  # take object as is
            new.append(obj)

        return new

    return _walk_json(obj, new={})

Here's how it's used for an overly simple json object:

def my_key_transform(x):
    return x.upper()

my_obj = {"a": 1, "b": 2, "c": 3}

result = walk_json(my_obj, key_transform=my_key_transform)

result
{"A": 1, "B": 2, "C": 3}

Answer 2

It can be painful navigating and modify deeply nested objects derived from JSON objects. In Functions that help to understand json(dict) structure I posted code that allows you to navigate such objects. Please read the explanation in that answer. In this answer, I'll show how you can use that code to modify the dictionary keys in such objects.

Briefly, find_key is a recursive generator that will find all the keys with a given name. You can use the next function to get the first (or only) matching name. Or call find_key in a for loop if you need to work with multiple keys that have the same name.

Each value yielded by find_key is a list of the dict keys and list indices need to reach the desired key.

from json import dumps

def find_key(obj, key):
    if isinstance(obj, dict):
        yield from iter_dict(obj, key, [])
    elif isinstance(obj, list):
        yield from iter_list(obj, key, [])

def iter_dict(d, key, indices):
    for k, v in d.items():
        if k == key:
            yield indices + [k], v
        if isinstance(v, dict):
            yield from iter_dict(v, key, indices + [k])
        elif isinstance(v, list):
            yield from iter_list(v, key, indices + [k])

def iter_list(seq, key, indices):
    for k, v in enumerate(seq):
        if isinstance(v, dict):
            yield from iter_dict(v, key, indices + [k])
        elif isinstance(v, list):
            yield from iter_list(v, key, indices + [k])


results = {
    "Game": 12345,
    "stats": [
        {
            "detail": [
                {
                    "goals": 4,
                    "refs": {
                        "number": 0
                    }
                }
            ]
        }
    ]
}

# Change oldkey to newkey      
oldkey, newkey = 'goals', 'goals_against'
# Find the first occurrence of the oldkey
seq, val = next(find_key(results, oldkey))
print('seq:', seq, 'val:', val)

# Get the object that contains the oldkey 
obj = results
for k in seq[:-1]:
    obj = obj[k]

# Change the key
obj[newkey] = obj.pop(oldkey)

print(dumps(results, indent=4))

output

seq: ['stats', 0, 'detail', 0, 'goals'] val: 4
{
    "Game": 12345,
    "stats": [
        {
            "detail": [
                {
                    "refs": {
                        "number": 0
                    },
                    "goals_against": 4
                }
            ]
        }
    ]
}