Using JSON keys as attributes in nested JSON

Question

I'm working with nested JSON-like data structures in python 2.7 that I exchange with some foreign perl code. I just want to 'work with' these nested structures of lists and dictionaries in amore pythonic way.

So if I have a structure like this...

a = {
    'x': 4,
    'y': [2, 3, { 'a': 55, 'b': 66 }],
}

...I want to be able to deal with it in a python script as if it was nested python classes/Structs, like this:

>>> aa = j2p(a)  # <<- this is what I'm after.
>>> print aa.x
4
>>> aa.z = 99
>>> print a
{
    'x': 4,
    'y': [2, 3, { 'a': 55, 'b': 66 }],
    'z': 99
}

>>> aa.y[2].b = 999

>>> print a
{
    'x': 4,
    'y': [2, 3, { 'a': 55, 'b': 999 }],
    'z': 99
}

Thus aa is a proxy into the original structure. This is what I came up with so far, inspired by the excellent What is a metaclass in Python? question.

def j2p(x):
    """j2p creates a pythonic interface to nested arrays and
    dictionaries, as returned by json readers.

    >>> a = { 'x':[5,8], 'y':5}
    >>> aa = j2p(a)
    >>> aa.y=7
    >>> print a
    {'x': [5, 8], 'y':7}
    >>> aa.x[1]=99
    >>> print a
    {'x': [5, 99], 'y':7}

    >>> aa.x[0] = {'g':5, 'h':9}
    >>> print a
    {'x': [ {'g':5, 'h':9} , 99], 'y':7}
    >>> print aa.x[0].g
    5
    """
    if isinstance(x, list):
        return _list_proxy(x)
    elif isinstance(x, dict):
        return _dict_proxy(x)
    else:
        return x

class _list_proxy(object):
    def __init__(self, proxied_list):
        object.__setattr__(self, 'data', proxied_list)
    def __getitem__(self, a):
        return j2p(object.__getattribute__(self, 'data').__getitem__(a))
    def __setitem__(self, a, v):
        return object.__getattribute__(self, 'data').__setitem__(a, v)


class _dict_proxy(_list_proxy):
    def __init__(self, proxied_dict):
        _list_proxy.__init__(self, proxied_dict)
    def __getattribute__(self, a):
        return j2p(object.__getattribute__(self, 'data').__getitem__(a))
    def __setattr__(self, a, v):
        return object.__getattribute__(self, 'data').__setitem__(a, v)


def p2j(x):
    """p2j gives back the underlying json-ic json-ic nested
    dictionary/list structure of an object or attribute created with
    j2p.
    """
    if isinstance(x, (_list_proxy, _dict_proxy)):
        return object.__getattribute__(x, 'data')
    else:
        return x

Now I wonder whether there is an elegant way of mapping a whole set of the __*__ special functions, like __iter__, __delitem__? so I don't need to unwrap things using p2j() just to iterate or do other pythonic stuff.

# today:
for i in p2j(aa.y):
     print i
# would like to...
for i in aa.y:
     print i

Answer 1

There is an attrdict library that does exactly that in a very safe manner, but if you want, a quick and dirty (possibly leaking memory) approach was given in this answer:

class AttrDict(dict):
    def __init__(self, *args, **kwargs):
        super(AttrDict, self).__init__(*args, **kwargs)
        self.__dict__ = self

j = '{"y": [2, 3, {"a": 55, "b": 66}], "x": 4}'
aa = json.loads(j, object_hook=AttrDict)

Answer 2

I think you're making this more complex than it needs to be. If I understand you correctly, all you should need to do is this:

import json

class Struct(dict):
    def __getattr__(self, name):
        return self[name]

    def __setattr__(self, name, value):
        self[name] = value

    def __delattr__(self, name):
        del self[name]

j = '{"y": [2, 3, {"a": 55, "b": 66}], "x": 4}'

aa = json.loads(j, object_hook=Struct)

for i in aa.y:
    print(i)

When you load JSON, the object_hook parameter lets you specify a callable object to process objects that it loads. I've just used it to turn the dict into an object that allows attribute access to its keys. Docs

Answer 3

I found the answer: There is intentionally no way to automatically map the special methods in python, using __getattribute__. So to achieve what I want, I need to explicitely define all special methods like __len__ one after the other.