Nested Dictionary Iteration in Python

Question

I have a dictionary that has a few options key value pairs and then a sub_dict attribute that is a list of more dictionaries with the same attributes and these also have the potential to have sub_dicts all the way down.

In python I break it apart and work on them individually and change them one at a time and want to recombine the changed system with the whole one. I am not sure how to iterate through it though.

{base_system: {
name: "root",
description: "data dictionary",
common_data: {},
other_data: {},
more_data: {},
sub_systems: [
    {
        base_system: {
        name: "another system",
        description: "inherits from top level",
        sub_systems: [
            {
                base_system: {}
            },
            {
                base_system: {}
            }
        ]
        }
    },
    {
        base_system: {
            name: "one more system",
            description: "inheriting again",
            sub_systems: [
            {
                base_system: {
                name: "child system",
                description: "no kids here",
                other_data: {},
                more_data: {}
                }
            },
            {
                base_system: {
                name: "kid system",
                description: "no children here"
                }
            }
            ]
        }
    }
]
}

}

I want to do something like this but I'm not sure what to do to make it recursive.

#Have some recursive function to dig through the overall dictionary then test:
if the_dict_object["space_system"]["name"] == changed_json_system["space_system"]["name"]:
  #then when it passes that if statement I can just set
  the_dict_object = changed_json_system

But I am not sure how to iterate through the nested dictionary and still have a hold of the overall object.

Answer 1

Here is example code that recursively walks through your dict structure. For this example, it replaces the description with the upper-case of the description.

_DESC = "description"
_BASESYS = "base_system"
_SUBSYS = "sub_systems"

def uppercase_desc(system_info):
    """
    Change a system object such that the description is in upper-case.
    """
    if _BASESYS not in system_info:
        return

    subd = system_info[_BASESYS]
    if _DESC in subd:
        subd[_DESC] = subd[_DESC].upper()
    if _SUBSYS not in subd:
        return
    for d in subd[_SUBSYS]:
        uppercase_desc(d)

if __name__ == "__main__":
    import json
    with open("data.json", "rt") as f:
        s = f.read()
    system_info = json.loads(s)

    uppercase_desc(system_info)
    s = json.dumps(system_info, indent=4, sort_keys=True)
    print(s)

The above code modifies the dictionary in-place. It's a little trickier, but not bad, to make a copy as you go and return the copy. This may be preferable.

The only tricky part here is that the code uses copy.deepcopy() by default. Since we don't know what might be in the dictionary, and we want to return a copy, we can just call copy.deepcopy() on everything; it will trivially work and do the right thing on simple objects like 3 (integer object of value 3).

import copy

_DESC = "description"
_BASESYS = "base_system"
_SUBSYS = "sub_systems"

def uppercase_desc(system_info):
    """
    Change a system object such that the description is in upper-case.
    """
    if _BASESYS not in system_info:
        raise ValueError("only works on a system info dict")

    # put in the base_system key and an empty subdir
    newsubd = {}
    new_system_info = {_BASESYS: newsubd}

    subd = system_info[_BASESYS]
    for key, value in subd.items():
        if _DESC == key:
            newsubd[key] = value.upper()
        elif _SUBSYS == key:
            newsubd[key] = [uppercase_desc(d) for d in value]
        else:
            newsubd[key] = copy.deepcopy(value)
    return new_system_info

if __name__ == "__main__":
    import json
    with open("data.json", "rt") as f:
        s = f.read()
    system_info = json.loads(s)

    new_system_info = uppercase_desc(system_info)
    s = json.dumps(new_system_info, indent=4, sort_keys=True)
    print(s)

P.S. The example data you posted isn't valid JSON. I modified it by putting double-quotes around the keys, and pretty-printed it with nice indents, to make my test file data.json. Here it is:

{
    "base_system": {
        "name": "root", 
        "description": "data dictionary", 
        "more_data": {}, 
        "common_data": {}, 
        "sub_systems": [
            {
                "base_system": {
                    "name": "another system", 
                    "sub_systems": [
                        {
                            "base_system": {}
                        }, 
                        {
                            "base_system": {}
                        }
                    ], 
                    "description": "inherits from top level"
                }
            }, 
            {
                "base_system": {
                    "name": "one more system", 
                    "sub_systems": [
                        {
                            "base_system": {
                                "more_data": {}, 
                                "other_data": {}, 
                                "name": "child system", 
                                "description": "no kids here"
                            }
                        }, 
                        {
                            "base_system": {
                                "name": "kid system", 
                                "description": "no children here"
                            }
                        }
                    ], 
                    "description": "inheriting again"
                }
            }
        ], 
        "other_data": {}
    }
}

Answer 2

You can use instanceof() method to check if something is a dict, then if true you can make your code to iterate through this dict. I would do a recursion in this case.

def read_dict(some_dictionary):
    for key, value in some_dictionary:

        if isinstance(value, dict): 
            # if value is another dict, iterate through the key,value pairs in value
            read_dict(value) 
        elif isinstance(value, list):
            # if value is a list, add your own code to iterate through a list
            pass
        else:
            #not a dict, do what you needed to do eg:
            print 'value of %s is %s' % (key, value) 

read_dict(the_dict_object)