How to convert a nested Python dict to object?

Question

I'm searching for an elegant way to get data using attribute access on a dict with some nested dicts and lists (i.e. javascript-style object syntax).

For example:

>>> d = {'a': 1, 'b': {'c': 2}, 'd': ["hi", {'foo': "bar"}]}

Should be accessible in this way:

>>> x = dict2obj(d)
>>> x.a
1
>>> x.b.c
2
>>> x.d[1].foo
bar

I think, this is not possible without recursion, but what would be a nice way to get an object style for dicts?

Answer 1

You can leverage the json module of the standard library with a custom object hook:

import json

class DictObject(object):
    def __init__(self, dict_):
        self.__dict__.update(dict_)

    @classmethod
    def from_dict(cls, d):
        return json.loads(json.dumps(d), object_hook=DictObject)

Example usage:

>>> d = {'a': 1, 'b': {'c': 2}, 'd': ['hi', {'foo': 'bar'}]}
>>> o = DictObject.from_dict(d)
>>> o.a
1
>>> o.b.c
2
>>> o.d[0]
'hi'
>>> o.d[1].foo
'bar'
>>>

And it is not strictly read-only as it is with namedtuple, i.e. you can change values – not structure:

>>> o.b.c = 3
>>> o.b.c
3

Answer 2

The exact solution of the question can be achieved easily by PyPI package named attrdict. The interesting fact about this package is that the dict can be accessed either as keys or as attributes. Here is the solution -

from attrdict import AttrDict

d = {'a': 1, 'b': {'c': 2}, 'd': ["hi", {'foo': "bar"}]}

x = AttrDict(d)

print(x.a, x['a'])
print(x.b.c, x['b']['c'])
print(x.d[1].foo, x['d'][1]['foo'])

And output is as follows (obviously with no error) -

1 1
2 2
bar bar

N.B. It was first released in Feb 2, 2019 which means at the time of asking this question, this third party pypi package didn't exist. But if someone now wants to access dict value either by key or by attribute, this package surely can help as magic with only one line of code.

Answer 3

The following code from here, works on nested dictionaries and IDEs such as VS Code are able to hint the existing attributes:

class Struct:
    def __init__(self, **kwargs):
        for key, value in kwargs.items():
            if isinstance(value, dict):
                self.__dict__[key] = Struct(**value)
            else:
                self.__dict__[key] = value


my_dict = {
    'name': 'bobbyhadz',
    'address': {
        'country': 'Country A',
        'city': 'City A',
        'codes': [1, 2, 3]
    },
}

obj = Struct(**my_dict)

If you want to see how to load a YAML file and covert it to a Python object, see this gist.

Answer 4

class obj(object):
    def __init__(self, d):
        for k, v in d.items():
            if isinstance(k, (list, tuple)):
                setattr(self, k, [obj(x) if isinstance(x, dict) else x for x in v])
            else:
                setattr(self, k, obj(v) if isinstance(v, dict) else v)

>>> d = {'a': 1, 'b': {'c': 2}, 'd': ["hi", {'foo': "bar"}]}
>>> x = obj(d)
>>> x.b.c
2
>>> x.d[1].foo
'bar'

Answer 5

Taking what I feel are the best aspects of the previous examples, here's what I came up with:

class Struct:
    """The recursive class for building and representing objects with."""

    def __init__(self, obj):
        for k, v in obj.items():
            if isinstance(v, dict):
                setattr(self, k, Struct(v))
            else:
                setattr(self, k, v)

    def __getitem__(self, val):
        return self.__dict__[val]

    def __repr__(self):
        return '{%s}' % str(', '.join('%s : %s' % (k, repr(v)) for (k, v) in self.__dict__.items()))

Answer 6

Looking for a simple wrapper class for dict enabling attribute-style key access/assignment (dot notation) I was not satisfied with the existing options for the reasons below.

dataclasses, pydantic, etc. are great but require a static definition of the content. Also, they cannot replace dict in code which relied on dict since they don't share the same methods and __getitem__() syntax is not supported.

Hence, I developed MetaDict. It behaves exactly like dict but enables dot notation and IDE autocompletion (if the object is loaded in the RAM) without the shortcomings and potential namespace conflicts of other solutions. All features and usage examples can be found on GitHub (see link above).

Full disclosure: I am the author of MetaDict.

Shortcomings/limitations I encountered when trying out other solutions:

• Addict
  • No key autocompletion in IDE
  • Nested key assignment cannot be turned off
  • Newly assigned dict objects are not converted to support attribute-style key access
  • Shadows inbuilt type Dict
• Prodict
  • No key autocompletion in IDE without defining a static schema (similar to dataclass)
  • No recursive conversion of dict objects when embedded in list or other inbuilt iterables
• AttrDict
  • No key autocompletion in IDE
  • Converts list objects to tuple behind the scenes
• Munch
  • Inbuilt methods like items(), update(), etc. can be overwritten with obj.items = [1, 2, 3]
  • No recursive conversion of dict objects when embedded in list or other inbuilt iterables
• EasyDict
  • Only strings are valid keys, but dict accepts all hashable objects as keys
  • Inbuilt methods like items(), update(), etc. can be overwritten with obj.items = [1, 2, 3]
  • Inbuilt methods don't behave as expected: obj.pop('unknown_key', None) raises an AttributeError

Note: I wrote a similar answer in this stackoverflow, which is related.

Answer 7

You can use my way to handle it.

somedict= {"person": {"name": "daniel"}}

class convertor:
    def __init__(self, dic: dict) -> object:
        self.dict = dic

        def recursive_check(obj):
            for key, value in dic.items():
                if isinstance(value, dict):
                    value= convertor(value)
                setattr(obj, key, value)
        recursive_check(self)
my_object= convertor(somedict)

print(my_object.person.name)

Answer 8

Surprisingly no one has mentioned Bunch. This library is exclusively meant to provide attribute style access to dict objects and does exactly what the OP wants. A demonstration:

>>> from bunch import bunchify
>>> d = {'a': 1, 'b': {'c': 2}, 'd': ["hi", {'foo': "bar"}]}
>>> x = bunchify(d)
>>> x.a
1
>>> x.b.c
2
>>> x.d[1].foo
'bar'

A Python 3 library is available at https://github.com/Infinidat/munch - Credit goes to codyzu

>>> from munch import DefaultMunch
>>> d = {'a': 1, 'b': {'c': 2}, 'd': ["hi", {'foo': "bar"}]}
>>> obj = DefaultMunch.fromDict(d)
>>> obj.b.c
2
>>> obj.a
1
>>> obj.d[1].foo
'bar'

Answer 9

class Dict2Obj:
    def __init__(self, json_data):
        self.convert(json_data)

    def convert(self, json_data):
        if not isinstance(json_data, dict):
            return
        for key in json_data:
            if not isinstance(json_data[key], dict):
                self.__dict__.update({key: json_data[key]})
            else:
                self.__dict__.update({ key: Dict2Obj(json_data[key])})

I could not find the implementation of nested dictionary to object, so wrote one.

Usage:

>>> json_data = {"a": {"b": 2}, "c": 3}
>>> out_obj = Dict2Obj(json_data)
>>> out_obj.a
<Dict2Obj object at 0x7f3dc22c2d68>
>>> out_obj.a.b
2
>>> out_obj.a.c
3

Answer 10

In 2021, use pydantic BaseModel - will convert nested dicts and nested json objects to python objects and vice versa:

https://pydantic-docs.helpmanual.io/usage/models/

>>> class Foo(BaseModel):
...     count: int
...     size: float = None
... 
>>> 
>>> class Bar(BaseModel):
...     apple = 'x'
...     banana = 'y'
... 
>>> 
>>> class Spam(BaseModel):
...     foo: Foo
...     bars: List[Bar]
... 
>>> 
>>> m = Spam(foo={'count': 4}, bars=[{'apple': 'x1'}, {'apple': 'x2'}])

Object to dict

>>> print(m.dict())
{'foo': {'count': 4, 'size': None}, 'bars': [{'apple': 'x1', 'banana': 'y'}, {'apple': 'x2', 'banana': 'y'}]}

Object to JSON

>>> print(m.json())
{"foo": {"count": 4, "size": null}, "bars": [{"apple": "x1", "banana": "y"}, {"apple": "x2", "banana": "y"}]}

Dict to object

>>> spam = Spam.parse_obj({'foo': {'count': 4, 'size': None}, 'bars': [{'apple': 'x1', 'banana': 'y'}, {'apple': 'x2', 'banana': 'y2'}]})
>>> spam
Spam(foo=Foo(count=4, size=None), bars=[Bar(apple='x1', banana='y'), Bar(apple='x2', banana='y2')])

JSON to object

>>> spam = Spam.parse_raw('{"foo": {"count": 4, "size": null}, "bars": [{"apple": "x1", "banana": "y"}, {"apple": "x2", "banana": "y"}]}')
>>> spam
Spam(foo=Foo(count=4, size=None), bars=[Bar(apple='x1', banana='y'), Bar(apple='x2', banana='y')])

Answer 11

# Applies to Python-3 Standard Library
class Struct(object):
    def __init__(self, data):
        for name, value in data.items():
            setattr(self, name, self._wrap(value))

    def _wrap(self, value):
        if isinstance(value, (tuple, list, set, frozenset)): 
            return type(value)([self._wrap(v) for v in value])
        else:
            return Struct(value) if isinstance(value, dict) else value


# Applies to Python-2 Standard Library
class Struct(object):
    def __init__(self, data):
        for name, value in data.iteritems():
            setattr(self, name, self._wrap(value))

    def _wrap(self, value):
        if isinstance(value, (tuple, list, set, frozenset)): 
            return type(value)([self._wrap(v) for v in value])
        else:
            return Struct(value) if isinstance(value, dict) else value

Can be used with any sequence/dict/value structure of any depth.

Answer 12

Updated with recursive array expansion on @max-sirwa 's code

class Objectify:
    def __init__(self, **kwargs):
        for key, value in kwargs.items():
            if isinstance(value, dict):
                f = Objectify(**value)
                self.__dict__.update({key: f})
            elif isinstance(value, list):
                t = []
                for i in value:
                    t.append(Objectify(**i)) if isinstance(i, dict) else t.append(i)
                self.__dict__.update({key: t})
            else:
                self.__dict__.update({key: value})

Answer 13

Building on what was done earlier by the accepted answer, if you would like to have it recursive.

class FullStruct:
    def __init__(self, **kwargs):
        for key, value in kwargs.items():
            if isinstance(value, dict):
                f = FullStruct(**value)
                self.__dict__.update({key: f})
            else:
                self.__dict__.update({key: value})

Answer 14

The simplest way would be using collections.namedtuple.

I find the following 4-liner the most beautiful, which supports nested dictionaries:

def dict_to_namedtuple(typename, data):
    return namedtuple(typename, data.keys())(
        *(dict_to_namedtuple(typename + '_' + k, v) if isinstance(v, dict) else v for k, v in data.items())
    )

The output will look good as well:

>>> nt = dict_to_namedtuple('config', {
...     'path': '/app',
...     'debug': {'level': 'error', 'stream': 'stdout'}
... })

>>> print(nt)
config(path='/app', debug=config_debug(level='error', stream='stdout'))

>>> print(nt.debug.level)
'error'

Answer 15

I wasn't satisfied with the marked and upvoted answers, so here is a simple and general solution for transforming JSON-style nested datastructures (made of dicts and lists) into hierachies of plain objects:

# tested in: Python 3.8
from collections import abc
from typings import Any, Iterable, Mapping, Union

class DataObject:
    def __repr__(self):
        return str({k: v for k, v in vars(self).items()})

def data_to_object(data: Union[Mapping[str, Any], Iterable]) -> object:
    """
    Example
    -------
    >>> data = {
    ...     "name": "Bob Howard",
    ...     "positions": [{"department": "ER", "manager_id": 13}],
    ... }
    ... data_to_object(data).positions[0].manager_id
    13
    """
    if isinstance(data, abc.Mapping):
        r = DataObject()
        for k, v in data.items():
            if type(v) is dict or type(v) is list:
                setattr(r, k, data_to_object(v))
            else:
                setattr(r, k, v)
        return r
    elif isinstance(data, abc.Iterable):
        return [data_to_object(e) for e in data]
    else:
        return data

Answer 16

Typically you want to mirror dict hierarchy into your object but not list or tuples which are typically at lowest level. So this is how I did this:

class defDictToObject(object):

    def __init__(self, myDict):
        for key, value in myDict.items():
            if type(value) == dict:
                setattr(self, key, defDictToObject(value))
            else:
                setattr(self, key, value)

So we do:

myDict = { 'a': 1,
           'b': { 
              'b1': {'x': 1,
                    'y': 2} },
           'c': ['hi', 'bar'] 
         }

and get:

x.b.b1.x 1

x.c ['hi', 'bar']

Answer 17

Convert dict to object

from types import SimpleNamespace

def dict2obj(data):
    """将字典对象转换为可访问的对象属性"""
    if not isinstance(data, dict):
        raise ValueError('data must be dict object.')

    def _d2o(d):
        _d = {}
        for key, item in d.items():
            if isinstance(item, dict):
                _d[key] = _d2o(item)
            else:
                _d[key] = item
        return SimpleNamespace(**_d)

    return _d2o(data)

Reference Answer

Answer 18

There's a collection helper called namedtuple, that can do this for you:

from collections import namedtuple

d_named = namedtuple('Struct', d.keys())(*d.values())

In [7]: d_named
Out[7]: Struct(a=1, b={'c': 2}, d=['hi', {'foo': 'bar'}])

In [8]: d_named.a
Out[8]: 1

Answer 19

I know there's already a lot of answers here already and I'm late to the party but this method will recursively and 'in place' convert a dictionary to an object-like structure... Works in 3.x.x

def dictToObject(d):
    for k,v in d.items():
        if isinstance(v, dict):
            d[k] = dictToObject(v)
    return namedtuple('object', d.keys())(*d.values())

# Dictionary created from JSON file
d = {
    'primaryKey': 'id', 
    'metadata': 
        {
            'rows': 0, 
            'lastID': 0
        }, 
    'columns': 
        {
            'col2': {
                'dataType': 'string', 
                'name': 'addressLine1'
            }, 
            'col1': {
                'datatype': 'string', 
                'name': 'postcode'
            }, 
            'col3': {
                'dataType': 'string', 
                'name': 'addressLine2'
            }, 
            'col0': {
                'datatype': 'integer', 
                'name': 'id'
            }, 
            'col4': {
                'dataType': 'string', 
                'name': 'contactNumber'
            }
        }, 
        'secondaryKeys': {}
}

d1 = dictToObject(d)
d1.columns.col1 # == object(datatype='string', name='postcode')
d1.metadata.rows # == 0

Answer 20

This also works well too

class DObj(object):
    pass

dobj = Dobj()
dobj.__dict__ = {'a': 'aaa', 'b': 'bbb'}

print dobj.a
>>> aaa
print dobj.b
>>> bbb

Answer 21

I ended up trying BOTH the AttrDict and the Bunch libraries and found them to be way too slow for my uses. After a friend and I looked into it, we found that the main method for writing these libraries results in the library aggressively recursing through a nested object and making copies of the dictionary object throughout. With this in mind, we made two key changes. 1) We made attributes lazy-loaded 2) instead of creating copies of a dictionary object, we create copies of a light-weight proxy object. This is the final implementation. The performance increase of using this code is incredible. When using AttrDict or Bunch, these two libraries alone consumed 1/2 and 1/3 respectively of my request time(what!?). This code reduced that time to almost nothing(somewhere in the range of 0.5ms). This of course depends on your needs, but if you are using this functionality quite a bit in your code, definitely go with something simple like this.

class DictProxy(object):
    def __init__(self, obj):
        self.obj = obj

    def __getitem__(self, key):
        return wrap(self.obj[key])

    def __getattr__(self, key):
        try:
            return wrap(getattr(self.obj, key))
        except AttributeError:
            try:
                return self[key]
            except KeyError:
                raise AttributeError(key)

    # you probably also want to proxy important list properties along like
    # items(), iteritems() and __len__

class ListProxy(object):
    def __init__(self, obj):
        self.obj = obj

    def __getitem__(self, key):
        return wrap(self.obj[key])

    # you probably also want to proxy important list properties along like
    # __iter__ and __len__

def wrap(value):
    if isinstance(value, dict):
        return DictProxy(value)
    if isinstance(value, (tuple, list)):
        return ListProxy(value)
    return value

See the original implementation here by https://stackoverflow.com/users/704327/michael-merickel.

The other thing to note, is that this implementation is pretty simple and doesn't implement all of the methods you might need. You'll need to write those as required on the DictProxy or ListProxy objects.

Answer 22

Update: In Python 2.6 and onwards, consider whether the namedtuple data structure suits your needs:

>>> from collections import namedtuple
>>> MyStruct = namedtuple('MyStruct', 'a b d')
>>> s = MyStruct(a=1, b={'c': 2}, d=['hi'])
>>> s
MyStruct(a=1, b={'c': 2}, d=['hi'])
>>> s.a
1
>>> s.b
{'c': 2}
>>> s.c
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'MyStruct' object has no attribute 'c'
>>> s.d
['hi']

The alternative (original answer contents) is:

class Struct:
    def __init__(self, **entries):
        self.__dict__.update(entries)

Then, you can use:

>>> args = {'a': 1, 'b': 2}
>>> s = Struct(**args)
>>> s
<__main__.Struct instance at 0x01D6A738>
>>> s.a
1
>>> s.b
2

Answer 23

What about just assigning your dict to the __dict__ of an empty object?

class Object:
    """If your dict is "flat", this is a simple way to create an object from a dict

    >>> obj = Object()
    >>> obj.__dict__ = d
    >>> d.a
    1
    """
    pass

Of course this fails on your nested dict example unless you walk the dict recursively:

# For a nested dict, you need to recursively update __dict__
def dict2obj(d):
    """Convert a dict to an object

    >>> d = {'a': 1, 'b': {'c': 2}, 'd': ["hi", {'foo': "bar"}]}
    >>> obj = dict2obj(d)
    >>> obj.b.c
    2
    >>> obj.d
    ["hi", {'foo': "bar"}]
    """
    try:
        d = dict(d)
    except (TypeError, ValueError):
        return d
    obj = Object()
    for k, v in d.iteritems():
        obj.__dict__[k] = dict2obj(v)
    return obj

And your example list element was probably meant to be a Mapping, a list of (key, value) pairs like this:

>>> d = {'a': 1, 'b': {'c': 2}, 'd': [("hi", {'foo': "bar"})]}
>>> obj = dict2obj(d)
>>> obj.d.hi.foo
"bar"

Answer 24

Here is a nested-ready version with namedtuple:

from collections import namedtuple

class Struct(object):
    def __new__(cls, data):
        if isinstance(data, dict):
            return namedtuple(
                'Struct', data.iterkeys()
            )(
                *(Struct(val) for val in data.values())
            )
        elif isinstance(data, (tuple, list, set, frozenset)):
            return type(data)(Struct(_) for _ in data)
        else:
            return data

=>

>>> d = {'a': 1, 'b': {'c': 2}, 'd': ["hi", {'foo': "bar"}]}
>>> s = Struct(d)
>>> s.d
['hi', Struct(foo='bar')]
>>> s.d[0]
'hi'
>>> s.d[1].foo
'bar'

Answer 25

This little class never gives me any problem, just extend it and use the copy() method:

  import simplejson as json

  class BlindCopy(object):

    def copy(self, json_str):
        dic = json.loads(json_str)
        for k, v in dic.iteritems():
            if hasattr(self, k):
                setattr(self, k, v);

Answer 26

Building off my answer to "python: How to add property to a class dynamically?":

class data(object):
    def __init__(self,*args,**argd):
        self.__dict__.update(dict(*args,**argd))

def makedata(d):
    d2 = {}
    for n in d:
        d2[n] = trydata(d[n])
    return data(d2)

def trydata(o):
    if isinstance(o,dict):
        return makedata(o)
    elif isinstance(o,list):
        return [trydata(i) for i in o]
    else:
        return o

You call makedata on the dictionary you want converted, or maybe trydata depending on what you expect as input, and it spits out a data object.

Notes:

• You can add elifs to trydata if you need more functionality.
• Obviously this won't work if you want x.a = {} or similar.
• If you want a readonly version, use the class data from the original answer.

Answer 27

If you want to access dict keys as an object (or as a dict for difficult keys), do it recursively, and also be able to update the original dict, you could do:

class Dictate(object):
    """Object view of a dict, updating the passed in dict when values are set
    or deleted. "Dictate" the contents of a dict...: """

    def __init__(self, d):
        # since __setattr__ is overridden, self.__dict = d doesn't work
        object.__setattr__(self, '_Dictate__dict', d)

    # Dictionary-like access / updates
    def __getitem__(self, name):
        value = self.__dict[name]
        if isinstance(value, dict):  # recursively view sub-dicts as objects
            value = Dictate(value)
        return value

    def __setitem__(self, name, value):
        self.__dict[name] = value
    def __delitem__(self, name):
        del self.__dict[name]

    # Object-like access / updates
    def __getattr__(self, name):
        return self[name]

    def __setattr__(self, name, value):
        self[name] = value
    def __delattr__(self, name):
        del self[name]

    def __repr__(self):
        return "%s(%r)" % (type(self).__name__, self.__dict)
    def __str__(self):
        return str(self.__dict)

Example usage:

d = {'a': 'b', 1: 2}
dd = Dictate(d)
assert dd.a == 'b'  # Access like an object
assert dd[1] == 2  # Access like a dict
# Updates affect d
dd.c = 'd'
assert d['c'] == 'd'
del dd.a
del dd[1]
# Inner dicts are mapped
dd.e = {}
dd.e.f = 'g'
assert dd['e'].f == 'g'
assert d == {'c': 'd', 'e': {'f': 'g'}}

Answer 28

I think a dict consists of number, string and dict is enough most time. So I ignore the situation that tuples, lists and other types not appearing in the final dimension of a dict.

Considering inheritance, combined with recursion, it solves the print problem conveniently and also provides two ways to query a data,one way to edit a data.

See the example below, a dict that describes some information about students:

group=["class1","class2","class3","class4",]
rank=["rank1","rank2","rank3","rank4","rank5",]
data=["name","sex","height","weight","score"]

#build a dict based on the lists above
student_dic=dict([(g,dict([(r,dict([(d,'') for d in data])) for r in rank ]))for g in group])

#this is the solution
class dic2class(dict):
    def __init__(self, dic):
        for key,val in dic.items():
            self.__dict__[key]=self[key]=dic2class(val) if isinstance(val,dict) else val


student_class=dic2class(student_dic)

#one way to edit:
student_class.class1.rank1['sex']='male'
student_class.class1.rank1['name']='Nan Xiang'

#two ways to query:
print student_class.class1.rank1
print student_class.class1['rank1']
print '-'*50
for rank in student_class.class1:
    print getattr(student_class.class1,rank)

Results:

{'score': '', 'sex': 'male', 'name': 'Nan Xiang', 'weight': '', 'height': ''}
{'score': '', 'sex': 'male', 'name': 'Nan Xiang', 'weight': '', 'height': ''}
--------------------------------------------------
{'score': '', 'sex': '', 'name': '', 'weight': '', 'height': ''}
{'score': '', 'sex': '', 'name': '', 'weight': '', 'height': ''}
{'score': '', 'sex': 'male', 'name': 'Nan Xiang', 'weight': '', 'height': ''}
{'score': '', 'sex': '', 'name': '', 'weight': '', 'height': ''}
{'score': '', 'sex': '', 'name': '', 'weight': '', 'height': ''}

Answer 29

This is another, alternative, way to convert a list of dictionaries to an object:

def dict2object(in_dict):
    class Struct(object):
        def __init__(self, in_dict):
            for key, value in in_dict.items():
                if isinstance(value, (list, tuple)):
                    setattr(
                        self, key,
                        [Struct(sub_dict) if isinstance(sub_dict, dict)
                         else sub_dict for sub_dict in value])
                else:
                    setattr(
                        self, key,
                        Struct(value) if isinstance(value, dict)
                        else value)
    return [Struct(sub_dict) for sub_dict in in_dict] \
        if isinstance(in_dict, list) else Struct(in_dict)

Answer 30

from mock import Mock
d = {'a': 1, 'b': {'c': 2}, 'd': ["hi", {'foo': "bar"}]}
my_data = Mock(**d)

# We got
# my_data.a == 1