Encoding nested python object in JSON

Question

I want to encode objects in JSON. But, I can not figure out how to make the output without the string escaping.

import json

class Abc:
    def __init__(self):
        self.name="abc name"
    def toJSON(self):
        return json.dumps(self.__dict__, cls=ComplexEncoder)

class Doc:
    def __init__(self):
        self.abc=Abc()
    def toJSON(self):
        return json.dumps(self.__dict__, cls=ComplexEncoder)

class ComplexEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, Abc) or isinstance(obj, Doc):
            return obj.toJSON()
        else:
            return json.JSONEncoder.default(self, obj)

doc=Doc()
print doc.toJSON()

The result is (the dumps returns a string representation, that's why the " are escaped)

{"abc": "{\"name\": \"abc name\"}"}

I want something a little bit different. The expected result is

{"abc": {"name": "abc name"}"}

But I don't see how to... Any hint ?

thanks in advance.

Answer 1

So, the immediate problem is that you're passing the json module a JSON value, which will get encoded as just another string in the JSON value.

The broader problem is that you're greatly overcomplicating this.

Drawing on JSON datetime between Python and JavaScript, I'd go with something closer to this:

import json

class Abc:
    def __init__(self):
        self.name="abc name"
    def jsonable(self):
        return self.name

class Doc:
    def __init__(self):
        self.abc=Abc()
    def jsonable(self):
        return self.__dict__

def ComplexHandler(Obj):
    if hasattr(Obj, 'jsonable'):
        return Obj.jsonable()
    else:
        raise TypeError('Object of type %s with value of %s is not JSON serializable' % (type(Obj), repr(Obj)))

doc=Doc()
print json.dumps(doc, default=ComplexHandler)

which gets you:

~$ python nestjson.py 
{"abc": "abc name"}
~$ 

This can be made cleaner/saner/safer (in particular, just grabbing __dict__ isn't generally a recommended thing to do outside debugging/troubleshooting), but it should get the point across. All you need, fundamentally, is a way to get a json-compatible object (whether that's a simple string or number, or a list or dict) out of each "node" in the tree. That object should not be an already-JSON-serialized object, which is what you were doing.

Answer 2

For more complex serialization I would use jsons, it was published in 2022.

• Turn Python objects into dicts or (JSON)strings and back

• No changes are required to your objects

• Easily customizable and extendable

• Works with data classes, attrs, and POPOs

    pip install jsons
    class Person:
          name:str
          birthday:datetime
  
    personObject = Person("Tony", date_of_birth)
  
    import jsons
    json_data = jsons.dumps(personObject, indent=4)
  

Answer 3

Although all the other solutions I assume they work I find they do have a lot of boilerplate code, when the goal is to only encode nested python objects.

In an article I found an elegant solution, which does exactly what you asked for but without the boilerplate code. As you can even have the de-serialization part for free as well I will show you first a solution to your exact question and then give a cleaner version where the de-serialization will work as well.

Exact solution to your question

import json


class Abc(object):
    def __init__(self):
        self.name = "abc name"


class Doc(object):
    def __init__(self):
        self.abc = Abc()


doc = Doc()

# Serialization
json_data = json.dumps(doc, default=lambda o: o.__dict__)
print(json_data)

This will output exactly what you where asking for:

{"abc": {"name": "abc name"}}

More elegant solution to enable serializing and de-seralizing

import json


class Abc(object):
    def __init__(self, name: str):
        self.name = name


class Doc(object):
    def __init__(self, abc):
        self.abc = abc


abc = Abc("abc name")
doc = Doc(abc)

# Serialization
json_data = json.dumps(doc, default=lambda o: o.__dict__)
print(json_data)

# De-serialization
decoded_doc = Doc(**json.loads(json_data))
print(decoded_doc)
print(vars(decoded_doc))

This will output the following:

{"abc": {"name": "abc name"}}
<__main__.Doc object at 0x7ff75366f250>
{'abc': {'name': 'abc name'}}

The whole magic works by defining a default lambda function: json_data = json.dumps(doc, default=lambda o: o.__dict__).

Answer 4

To avoid repetition of code like in Fred Laurent's answer I overloaded the __iter__() method as follows. This also permits to 'jsonize' list elements, datetime and decimal with no extra dependencies, just use dict().

import datetime
import decimal


class Jsonable(object):
    def __iter__(self):
        for attr, value in self.__dict__.iteritems():
            if isinstance(value, datetime.datetime):
                iso = value.isoformat()
                yield attr, iso
            elif isinstance(value, decimal.Decimal):
                yield attr, str(value)
            elif(hasattr(value, '__iter__')):
                if(hasattr(value, 'pop')):
                    a = []
                    for subval in value:
                        if(hasattr(subval, '__iter__')):
                            a.append(dict(subval))
                        else:
                            a.append(subval)
                    yield attr, a
                else:
                    yield attr, dict(value)
            else:
                yield attr, value

class Identity(Jsonable):
    def __init__(self):
        self.name="abc name"
        self.first="abc first"
        self.addr=Addr()

class Addr(Jsonable):
    def __init__(self):
        self.street="sesame street"
        self.zip="13000"

class Doc(Jsonable):
    def __init__(self):
        self.identity=Identity()
        self.data="all data"


def main():
    doc=Doc()
    print "-Dictionary- \n"
    print dict(doc)
    print "\n-JSON- \n"
    print json.dumps(dict(doc), sort_keys=True, indent=4)

if __name__ == '__main__':
    main()

The output:

-Dictionary- 

{'data': 'all data', 'identity': {'first': 'abc first', 'addr': {'street': 'sesame street', 'zip': '13000'}, 'name': 'abc name'}}

-JSON- 

{
    "data": "all data", 
    "identity": {
        "addr": {
            "street": "sesame street", 
            "zip": "13000"
        }, 
        "first": "abc first", 
        "name": "abc name"
    }
}

Hope it helps! Thanks

Answer 5

This is what you're looking for: https://github.com/jsonpickle/jsonpickle

It does nested serialization of Python objects and can easily be extended to serialize custom types.

Answer 6

I could not add this as a comment and adding as answer. Fred's final sample was useful for me.I was told jsonpickle does this, but could not get the module to install and run properly. So used the code here. Minor tweak though, I had way too many variables to add by hand to some of the objects. So this little loop simplified things:

def reprJSON(self):
    d = dict()
    for a, v in self.__dict__.items():
        if (hasattr(v, "reprJSON")):
            d[a] = v.reprJSON()
        else:
            d[a] = v
    return d

It can be used in any object that has a subclass that is too busy to hand encode. Or can be made a helper for all classes. This also works for the full JSON presentation of member arrays that contain other classes (as long as they implement reprJSON() of course).

Answer 7

my previous sample, with another nested object and your advices :

import json

class Identity:
    def __init__(self):
        self.name="abc name"
        self.first="abc first"
        self.addr=Addr()
    def reprJSON(self):
        return dict(name=self.name, firstname=self.first, address=self.addr) 

class Addr:
    def __init__(self):
        self.street="sesame street"
        self.zip="13000"
    def reprJSON(self):
        return dict(street=self.street, zip=self.zip) 

class Doc:
    def __init__(self):
        self.identity=Identity()
        self.data="all data"
    def reprJSON(self):
        return dict(id=self.identity, data=self.data) 

class ComplexEncoder(json.JSONEncoder):
    def default(self, obj):
        if hasattr(obj,'reprJSON'):
            return obj.reprJSON()
        else:
            return json.JSONEncoder.default(self, obj)

doc=Doc()
print "Str representation"
print doc.reprJSON()
print "Full JSON"
print json.dumps(doc.reprJSON(), cls=ComplexEncoder)
print "Partial JSON"
print json.dumps(doc.identity.addr.reprJSON(), cls=ComplexEncoder)

produces the expected result :

Str representation
{'data': 'all data', 'id': <__main__.Identity instance at 0x1005317e8>}
Full JSON
{"data": "all data", "id": {"name": "abc name", "firstname": "abc first", "address": {"street": "sesame street", "zip": "13000"}}}
Partial JSON
{"street": "sesame street", "zip": "13000"}

Thanks.