Serialisation of dynamic Python Dictionary Keys

Question

Problem

I am trying to map a response object into a Python class representation, but there are a number of keys within the response that are dynamic and therefore I am unable to map them explicitly to class members.

The response object:

{
   'rows': [
     {
       '1900000084913': '222222',
       '360018501198': '4003112',
       'custom_fields': [
         {'id': 360018501198, 'value': '4003112'},
         {'id': 1900000084913, 'value': '222222'}
        ]
     }
   ]
}
      

Within the object: '1900000084913' and '360018501198' are dynamically set. (At the moment I have added x and y as placeholders in the Row object)

Code:

from dataclasses import dataclass
from typing import List


@dataclass
class Serialisable:
  
    @classmethod
    def from_dict(cls, d):
        if d is not None:
            return cls(**d)


@dataclass
class CustomField(Serialisable):
    id: int
    value: str


@dataclass
class Row(Serialisable):
    x: str   # '1900000084913'  - How do I map these?
    y: str   # '360018501198'   -
    custom_fields: List[CustomField]

    @classmethod
    def from_dict(cls, d):
        if d is not None:
            kwargs = dict(d)
            custom_fields = kwargs.pop("custom_fields", None)
            if custom_fields is not None:
                kwargs["custom_fields"] = [
                    CustomField.from_dict(field) for field in custom_fields
                ]

            return cls(**kwargs)


@dataclass
class ResponseObject(Serialisable):
    rows: List[Row]

    @classmethod
    def from_dict(cls, d):
        if d is not None:
            kwargs = dict(d)
            rows = kwargs.pop("rows", None)
            if rows is not None:
                kwargs["rows"] = [
                    Row.from_dict(row) for row in rows
                ]

            return cls(**kwargs)


if __name__ == "__main__":
    response = {
        'rows': [
            {
                '1900000084913': '222222',
                '360018501198': '4003112',
                'custom_fields': [
                    {'id': 360018501198, 'value': '4003112'},
                    {'id': 1900000084913, 'value': '222222'}
                ]
            }
        ]
    }
    response_obj = ResponseObject.from_dict(response)
    print(response_obj)

If the keys are changed to x and y then this will map accordingly.

Answer 1

Assuming you are able to make a few assumptions:

• The dynamic field names will generally be numeric (can be cast to int if needed)
• You want x to map to the first dynamic key that appears in the dictinoary object, and y to map to the second dynamic key in the object.

Here is one solution that could work based on this:

from dataclasses import dataclass
from typing import List, Dict

d = {
   'rows': [
     {
       '1900000084913': '222222',
       '360018501198': '4003112',
       'custom_fields': [
         {'id': 360018501198, 'value': '4003112'},
         {'id': 1900000084913, 'value': '222222'}
        ]
     }
   ]
}


# @dataclass
class SerializableMixin:

    @classmethod
    def from_dict(cls, d: Dict):
        if d is not None:
            return cls(**d)


@dataclass
class CustomField(SerializableMixin):
    id: int
    value: str


@dataclass(init=False)
class Row:

    x: str   # '1900000084913'  - How do I map these?
    y: str   # '360018501198'   -
    custom_fields: List[CustomField]

    def __init__(self, custom_fields, **kwargs):
        self.custom_fields = [CustomField.from_dict(cf) for cf in custom_fields]

        placeholder_attrs = ['x', 'y']

        for key, value in kwargs.items():
            if key.isnumeric():
                attr_to_set = placeholder_attrs.pop(0)
                setattr(self, attr_to_set, value)


r = Row(**d["rows"][0])
print(r)

# prints:
#   Row(x='222222', y='4003112', custom_fields=[CustomField(id=360018501198, value='4003112'), CustomField(id=1900000084913, value='222222')])

Answer 2

In your case this will not work since your dynamical attributes are integers (or start with an integer number, even though represented as strings). You can't set attributes that start with an integer.

In any other case, maybe try something like the following:

@dataclass(init=False)
class Row(Serialisable):
    def __init__(self, custom_fields: List[CustomField], **kwargs):
        self.custom_fields = custom_fields
        for key, value in kwargs.items():
            setattr(self, key, value)

row = Row(custom_fields=[], dynamic_attribute_1="1", dynamic_attribute_2="2")

Accessing e.g. row.dynamic_attribute_1 will then work

>>> row.dynamic_attribute_1
'1'

Edit: Nevermind, it will also work in your case. At least the instantiation of the object. However, you will not be able to access the instance's attributes via row.123123 since it will raise a SyntaxError, but only by using the getattr method, i.e. getattr(row, "123123").

Answer 3

You have to create your own __init__ method and dynamically set using setattr. Writing your own __init__ also means you have write a __repr__ for printing as well. Keep in mind when you write your own __init__ some dataclass functionalities might not work as intended.

Simplified sample:

data = {
   'rows': [
     {
       '1900000084913': '222222',
       '360018501198': '4003112',
       'custom_fields': [
         {'id': 360018501198, 'value': '4003112'},
         {'id': 1900000084913, 'value': '222222'}
        ]
     }
   ]
}


class Row:
    def __init__(self, custom_fields, **kwargs):
        self.custom_fields = custom_fields

        for key, value in kwargs.items():
            setattr(self, key, value)

    def __repr__(self):
        items = ("%s=%r" % (k, v) for k, v in self.__dict__.items())
        return "%s(%s)" % (self.__class__.__name__, ', '.join(items))

r = Row(**data["rows"][0])
print(r)

You might also want to consider subclassing dict if you want to have a dynamic structure.