How to instantiate objects from different classes from a list of dictionaries?

Question

From an external webservice I receive a JSON response that looks like this (for convenience, it is already deserialized below):

alist = [
    {
        'type': 'type1',
        'name': 'dummy',
        'oid': 'some_id'
    },
    {
        'type': 'type2',
        'name': 'bigdummy',
        'anumber': 10
    }
]

For each type there exists a class. What I want is to instantiate objects of the respective class without using a long list of if-elifs.

I tried as follows:

Define classes A (for type1) and class B (for type2) using a from_dict classmethod:

class A():
    def __init__(self, name, oid, optional_stuff=None):
        self.name = name
        self.oid = oid
        self.optional_stuff = optional_stuff

    @classmethod
    def from_dict(cls, d):
        name = d['name']
        oid = d['oid']
        optional_stuff = d.get('optional_stuff')

        return cls(name, oid, optional_stuff)

    def foo(self):
        print('i am class A')


class B():
    def __init__(self, name, anumber):
        self.name = name
        self.number = anumber

    @classmethod
    def from_dict(cls, d):
        name = d['name']
        anumber = d['anumber']

        return cls(name, anumber)

Then I define a mapping dictionary:

string_class_map = {
    'type1': A,
    'type2': B
}

and finally convert alist to something the from_dict functions can easily consume:

alist2 = [
    {
        di['type']: {k: v for k, v in di.items() if k != 'type'}
    }
    for di in alist
]

[{'type1': {'name': 'dummy', 'oid': 'some_id'}},
 {'type2': {'name': 'bigdummy', 'anumber': 10}}]

object_list = [
    string_class_map[k].from_dict(v) for d in alist2 for k, v in d.items()
]

That gives me the desired output; when I do:

a = object_list[0]
a.name

will indeed print 'dummy'.

Question is whether there is a better way of getting from alist (this input I cannot change) to object_list.

Answer 1

Based on your feedback to my first answer, here's another — completely different — one that would allow you create however many more-or-less independent classes as you wish very easily. I think it's an improvement over than @Karl Knechtel's answer because there's no need to have a decorator and use it to "register" each of the subclasses — that effectively happens automatically by deriving each of them from a common base class.

Basically it's just an adaption of the pattern I used in my answer to the question:

    Improper use of __new__ to generate classes?

class Base:
    class Unknown(Exception): pass

    @classmethod
    def _get_all_subclasses(cls):
        """ Recursive generator of all class' subclasses. """
        for subclass in cls.__subclasses__():
            yield subclass
            for subclass in subclass._get_all_subclasses():
                yield subclass

    def __new__(cls, d):
        """ Create instance of appropriate subclass using type id. """
        type_id = d['type']
        for subclass in cls._get_all_subclasses():
            if subclass.type_id == type_id:
                # Using "object" base class method avoids recursion here.
                return object.__new__(subclass)
        else:  # No subclass with matching type_id found.
            raise Base.Unknown(f'type: {type_id!r}')

    def __repr__(self):
        return f'<{self.__class__.__name__} instance>'


class A(Base):
    type_id = 'type1'

    def __init__(self, d):
        self.name = d['name']
        self.oid = d['oid']
        self.optional_stuff = d.get('optional_stuff')

    def foo(self):
        print('I am class A')


class B(Base):
    type_id = 'type2'

    def __init__(self, d):
        self.name = d['name']
        self.anumber = d['anumber']


alist = [
    {
        'type': 'type1',
        'name': 'dummy',
        'oid': 'some_id'
    },
    {
        'type': 'type2',
        'name': 'bigdummy',
        'anumber': 10
    }
]


object_list = [Base(obj) for obj in alist]
print(f'object_list: {object_list}') # -> [<A instance>, <B instance>]
a = object_list[0]
print(repr(a.name))  # -> 'dummy'
b = object_list[1]
print(repr(b.name))  # -> 'bigdummy'

If you're using Python 3.6+, a more succinct implementation is possible using the object.__init_subclass__() classmethod which was added in that version:

class Base:
    _registry = {}

    @classmethod
    def __init_subclass__(cls, **kwargs):
        type_id = kwargs.pop('type_id', None)
        super().__init_subclass__(**kwargs)
        if type_id is not None:
            cls._registry[type_id] = cls

    def __new__(cls, d):
        """ Create instance of appropriate subclass. """
        type_id = d['type']
        subclass = Base._registry[type_id]
        return object.__new__(subclass)

    def __repr__(self):
        return f'<{self.__class__.__name__} instance>'


class A(Base, type_id='type1'):
    def __init__(self, d):
        self.name = d['name']
        self.oid = d['oid']
        self.optional_stuff = d.get('optional_stuff')

    def foo(self):
        print('I am class A')


class B(Base, type_id='type2'):
    def __init__(self, d):
        self.name = d['name']
        self.anumber = d['anumber']


alist = [
    {
        'type': 'type1',
        'name': 'dummy',
        'oid': 'some_id'
    },
    {
        'type': 'type2',
        'name': 'bigdummy',
        'anumber': 10
    }
]


object_list = [Base(obj) for obj in alist]
print(f'object_list: {object_list}') # -> [<A instance>, <B instance>]
a = object_list[0]
print(repr(a.name))  # -> 'dummy'
b = object_list[1]
print(repr(b.name))  # -> 'bigdummy'

Answer 2

• As long as the parameter names match up exactly, you don't need the from_dict classmethods - although you might still prefer to work through them as a place to add extra error handling. All we do is use argument unpacking.

• I would wrap up the process of creating a single object, first. Which is to say, a single "from_dict"-y method should handle the determination of the type, preparing the dict of the other parameters, and invoking the class factory.

• It seems useful to have a base class for these classes created from the factory - they at least have in common that they can be created this way, after all; you could add debugging stuff at that level; and it's a convenient place for the factory logic itself.

• You can use a decorator or metaclass to take care of the creation of the lookup map, to avoid having a separate chunk of data to maintain.

Putting that together, I get:

class JsonLoadable:
    _factory = {}


    def __str__(self):
        return f'{self.__class__.__name__}(**{{{self.__dict__}}})'


    @staticmethod # this is our decorator.
    def register(cls):
        # We use the class' __name__ attribute to get the lookup key.
        # So as long as the classes are named to match the JSON, this
        # automatically builds the correct mapping.
        JsonLoadable._factory[cls.__name__] = cls
        return cls


    @staticmethod
    def from_dict(d):
        d = d.copy()
        cls = JsonLoadable._factory[d.pop('type')]
        # this is the magic that lets us avoid class-specific logic.
        return cls(**d) 


# I'm pretty sure there's a way to streamline this further with metaclasses,
# but I'm not up to figuring it out at the moment...
@JsonLoadable.register
class A(JsonLoadable):
    def __init__(self, name, oid, optional_stuff=None):
        self.name = name
        self.oid = oid
        self.optional_stuff = optional_stuff


@JsonLoadable.register
class B(JsonLoadable):
    def __init__(self, name, anumber):
        self.name = name
        self.number = anumber


# And now our usage is simple:
objects = [JsonLoadable.from_dict(d) for d in alist]

Answer 3

Instead of writing a custom class for every 'type' that might be encounterd in alist, it seems like it would be simpler to use a generic class that would allow you to access their attributes (which is all you do with your sample classes).

To accomplish this the code below defines a subclass of the built-in dict class that will allow the value in it to be accessed as though they were instance attributes.

Here's what I mean:

from collections import Iterable, Mapping
from pprint import pprint, pformat


class AttrDict(dict):
    def __init__(self, d):
        for k, v in d.items():
            if isinstance(v, Mapping):
                d[k] = AttrDict(v)
            elif isinstance(v, Iterable) and not isinstance(v, str):
                d[k] = [AttrDict(x) if isinstance(x, Mapping) else x
                            for x in v]
        self.__dict__.update(d)

    def __repr__(self):
        return 'AttrDict({})'.format(repr(self.__dict__))


alist = [
    {
        'type': 'type1',
        'name': 'dummy',
        'oid': 'some_id'
    },
    {
        'type': 'type2',
        'name': 'bigdummy',
        'anumber': 10
    }
]

object_list = [AttrDict(obj) for obj in alist]
pprint(object_list)
print()  # -> [AttrDict({'type': 'type1', 'name': 'dummy', 'oid': 'some_id'}),
         # ->  AttrDict({'type': 'type2', 'name': 'bigdummy', 'anumber': 10})]


a = object_list[0]
print(repr(a.name))  # -> 'dummy'