CODEWITHSUNDEEP
jsonpython-3.xsubclass
Matt
Matt

Reputation: 189

How do I make a dict subclass json serializable?

I can represent the my Simple_Dict_Subclass and List_Subclass with json.dumps, but not Custom_Dict_Subclass. When json.dumps is called on List_Subclass its __iter__ method is called, so I reasoned that json.dumps would call a dictionary's items method. And items is called in Simple_Dict_Subclass but not Custom_Dict_Subclass. How can I make my Custom_Dict_Subclass json serializable like Simple_Dict_Subclass?

import json

class Custom_Dict_Subclass(dict):
    def __init__(self):
        self.data = {}

    def __setitem__(self, key, value):
        self.data[key] = value

    def __getitem__(self, key):
        return self.data[key]

    def __str__(self):
        return str(self.data)

    def items(self):
        print("'Items' called from Custom_Dict_Subclass")
        yield from self.data.items()

class Simple_Dict_Subclass(dict):
    def __setitem__(self, key, value):
        super().__setitem__(key, value)

    def __getitem__(self, key):
        return super().__getitem__(key)

    def __str__(self):
        return super().__str__()

    def items(self):
        print("'Items' called from Simple_Dict_Subclass")
        yield from super().items()

class List_Subclass(list):
    def __init__(self):
        self.data = []

    def __setitem__(self, index, value):
        self.data[index] = value

    def __getitem__(self, index):
        return self.data[index]

    def __str__(self):
        return str(self.data)

    def __iter__(self):
        yield from self.data

    def append(self, value):
        self.data.append(value)

d = Custom_Dict_Subclass()
d[0] = None
print(d)             # Works
print(json.dumps(d)) # Does't work

d = Simple_Dict_Subclass()
d[0] = None
print(d)             # Works
print(json.dumps(d)) # Works

l = List_Subclass()
l.append(None)
print(l)             # Works
print(json.dumps(l)) # Works

Output:

{0: None}   # Custom dict string     working
{}          # Custom dict json.dumps not working

{0: None}   # Simple dict string     working
'Items' called from Simple_Dict_Subclass
{"0": null} # Simple dict json.dumps working

[None]      # List string            working
[null]      # List json.dumps        working

Upvotes: 2

Views: 1266

Answers (2)

Alexander Ljungberg
Alexander Ljungberg

Reputation: 6362

The proposed solution of using a custom json.JSONEncoder is pragmatic but really just works around what IMHO looks like a bug in cpython. You are supposed to be able to subclass dict, right?

But the C optimised JSON encoder doesn't seem to know that. We find this code:

    if (PyDict_GET_SIZE(dct) == 0)  /* Fast path */
        return _PyUnicodeWriter_WriteASCIIString(writer, "{}", 2);

PyDict_GET_SIZE reads directly from the native dict structure which isn't under your direct control. If you want an entirely custom storage like in your Custom_Dict_Subclass it seems like you're out of luck, at least as of Python 3.12. (Incidentally, the cpython provided OrderedDict subclass works okay despite this because it uses the native storage, via super.)

If performance is not a concern, you could simply disable the C based JSON encoder: json.encoder.c_make_encoder = None.

Upvotes: 0

qouify
qouify

Reputation: 3900

Generally speaking, it is not safe to assume that json.dumps will trigger the items method of the dictionary. This is how it is implemented but you cannot rely on that.

In your case, the Custom_Dict_Subclass.items is never called because (key, value) pairs are not added to the dict object but to its data attribute.

To fix that you need to invoke the super methods in Custom_Dict_Subclass:

class Custom_Dict_Subclass(dict):
    def __init__(self):
        dict.__init__(self)
        self.data = {}
    def __setitem__(self, key, value):
        self.data[key] = value
        super().__setitem__(key, value)

The object is dumped correctly, but of course, (key, value) will then be stored twice: in the dict object and in its data attribute.

In that situation, it is better to define a sub class of json.JSONEncoder to implement the translation of a Custom_Dict_Subclass object to a json serialisable object and to give this class as the keyword argument cls of json.dumps:

import json

class Custom_Dict_Subclass:
    def __init__(self):
        self.data = {}

    def __setitem__(self, key, value):
        self.data[key] = value

    def __getitem__(self, key):
        return self.data[key]

    def __str__(self):
        return str(self.data)

    def items(self):
        print("'Items' called from Custom_Dict_Subclass")
        yield from self.data.items()

class CustomDictEncoder(json.JSONEncoder):
    def default(self, obj):
        """called by json.dumps to translate an object obj to
        a json serialisable data"""
        if isinstance(obj, Custom_Dict_Subclass):
            return obj.data
        return json.JSONEncoder.default(self, obj)  

d = Custom_Dict_Subclass()
d[0] = None
print(json.dumps(d, cls=CustomDictEncoder))

Upvotes: 0

Related Questions