Dynamically subclass an Enum base class

Question

I've set up a metaclass and base class pair for creating the line specifications of several different file types I have to parse.

I have decided to go with using enumerations because many of the individual parts of the different lines in the same file often have the same name. Enums make it easy to tell them apart. Additionally, the specification is rigid and there will be no need to add more members, or extend the line specifications later.

The specification classes work as expected. However, I am having some trouble dynamically creating them:

>>> C1 = LineMakerMeta('C1', (LineMakerBase,), dict(a = 0))
AttributeError: 'dict' object has no attribute '_member_names'

Is there a way around this? The example below works just fine:

class A1(LineMakerBase):
    Mode = 0, dict(fill=' ', align='>', type='s')
    Level = 8, dict(fill=' ', align='>', type='d')
    Method = 10, dict(fill=' ', align='>', type='d')
    _dummy = 20 # so that Method has a known length

A1.format(**dict(Mode='DESIGN', Level=3, Method=1))
# produces '  DESIGN 3         1'

The metaclass is based on enum.EnumMeta, and looks like this:

import enum

class LineMakerMeta(enum.EnumMeta):
    "Metaclass to produce formattable LineMaker child classes."
    def _iter_format(cls):
        "Iteratively generate formatters for the class members."
        for member in cls:
            yield member.formatter
    def __str__(cls):
        "Returns string line with all default values."
        return cls.format()
    def format(cls, **kwargs):
        "Create formatted version of the line populated by the kwargs members."
        # build resulting string by iterating through members
        result = ''
        for member in cls:
            # determine value to be injected into member
            try:
                try:
                    value = kwargs[member]
                except KeyError:
                    value = kwargs[member.name]
            except KeyError:
                value = member.default
            value_str = member.populate(value)
            result = result + value_str
        return result

And the base class is as follows:

class LineMakerBase(enum.Enum, metaclass=LineMakerMeta):
    """A base class for creating Enum subclasses used for populating lines of a file.

    Usage:

    class LineMaker(LineMakerBase):
        a = 0,      dict(align='>', fill=' ', type='f'), 3.14
        b = 10,     dict(align='>', fill=' ', type='d'), 1
        b = 15,     dict(align='>', fill=' ', type='s'), 'foo'
        #   ^-start ^---spec dictionary                  ^--default
    """
    def __init__(member, start, spec={}, default=None):
        member.start = start
        member.spec = spec
        if default is not None:
            member.default = default
        else:
            # assume value is numerical for all provided types other than 's' (string)
            default_or_set_type = member.spec.get('type','s')
            default = {'s': ''}.get(default_or_set_type, 0)
            member.default = default
    @property
    def formatter(member):
        """Produces a formatter in form of '{0:<format>}' based on the member.spec
        dictionary. The member.spec dictionary makes use of these keys ONLY (see
        the string.format docs):
            fill align sign width grouping_option precision type"""
        try:
            # get cached value
            return '{{0:{}}}'.format(member._formatter)
        except AttributeError:
            # add width to format spec if not there
            member.spec.setdefault('width', member.length if member.length != 0 else '')
            # build formatter using the available parts in the member.spec dictionary
            # any missing parts will simply not be present in the formatter
            formatter = ''
            for part in 'fill align sign width grouping_option precision type'.split():
                try:
                    spec_value = member.spec[part]
                except KeyError:
                    # missing part
                    continue
                else:
                    # add part
                    sub_formatter = '{!s}'.format(spec_value)
                    formatter = formatter + sub_formatter
            member._formatter = formatter
            return '{{0:{}}}'.format(formatter)
    def populate(member, value=None):
        "Injects the value into the member's formatter and returns the formatted string."
        formatter = member.formatter
        if value is not None:
            value_str = formatter.format(value)
        else:
            value_str = formatter.format(member.default)
        if len(value_str) > len(member) and len(member) != 0:
            raise ValueError(
                    'Length of object string {} ({}) exceeds available'
                    ' field length for {} ({}).'
                    .format(value_str, len(value_str), member.name, len(member)))
        return value_str
    @property
    def length(member):
        return len(member)
    def __len__(member):
        """Returns the length of the member field. The last member has no length.
        Length are based on simple subtraction of starting positions."""
        # get cached value
        try:
            return member._length
        # calculate member length
        except AttributeError:
            # compare by member values because member could be an alias
            members = list(type(member))
            try:
                next_index = next(
                        i+1
                        for i,m in enumerate(type(member))
                        if m.value == member.value
                        )
            except StopIteration:
                raise TypeError(
                       'The member value {} was not located in the {}.'
                       .format(member.value, type(member).__name__)
                       )
            try:
                next_member = members[next_index]
            except IndexError:
                # last member defaults to no length
                length = 0
            else:
                length = next_member.start - member.start
            member._length = length
            return length

Answer 1

The simplest way to create Enum subclasses on the fly is using Enum itself:

>>> from enum import Enum
>>> MyEnum = Enum('MyEnum', {'a': 0})
>>> MyEnum
<enum 'MyEnum'>
>>> MyEnum.a
<MyEnum.a: 0>
>>> type(MyEnum)
<class 'enum.EnumMeta'>

As for your custom methods, it might be simpler if you used regular functions, precisely because Enum implementation is so special.

Answer 2

Create your LineMakerBase class, and then use it like so:

C1 = LineMakerBase('C1', dict(a=0))

The metaclass was not meant to be used the way you are trying to use it. Check out this answer for advice on when metaclass subclasses are needed.

---

Some suggestions for your code:

the double try/except in format seems clearer as:

    for member in cls:
        if member in kwargs:
            value = kwargs[member]
        elif member.name in kwargs:
            value = kwargs[member.name]
        else:
            value = member.default

this code:

# compare by member values because member could be an alias
members = list(type(member))

1. would be clearer with list(member.__class__)
2. has a false comment: listing an Enum class will never include the aliases (unless you have overridden that part of EnumMeta)

instead of the complicated __len__ code you have now, and as long as you are subclassing EnumMeta you should extend __new__ to automatically calculate the lengths once:

# untested
def __new__(metacls, cls, bases, clsdict):
    # let the main EnumMeta code do the heavy lifting
    enum_cls = super(LineMakerMeta, metacls).__new__(cls, bases, clsdict)
    # go through the members and calculate the lengths
    canonical_members = [
           member
           for name, member in enum_cls.__members__.items()
           if name == member.name
           ]
    last_member = None
    for next_member in canonical_members:
        next_member.length = 0
        if last_member is not None:
            last_member.length = next_member.start - last_member.start

Answer 3

This line:

C1 = enum.EnumMeta('C1', (), dict(a = 0))

fails with exactly the same error message. The __new__ method of EnumMeta expects an instance of enum._EnumDict as its last argument. _EnumDict is a subclass of dict and provides an instance variable named _member_names, which of course a regular dict doesn't have. When you go through the standard mechanism of enum creation, this all happens correctly behind the scenes. That's why your other example works just fine.

This line:

C1 = enum.EnumMeta('C1', (), enum._EnumDict())

runs with no error. Unfortunately, the constructor of _EnumDict is defined as taking no arguments, so you can't initialize it with keywords as you apparently want to do.

In the implementation of enum that's backported to Python3.3, the following block of code appears in the constructor of EnumMeta. You could do something similar in your LineMakerMeta class:

def __new__(metacls, cls, bases, classdict):
    if type(classdict) is dict:
        original_dict = classdict
        classdict = _EnumDict()
        for k, v in original_dict.items():
            classdict[k] = v

In the official implementation, in Python3.5, the if statement and the subsequent block of code is gone for some reason. Therefore classdict must be an honest-to-god _EnumDict, and I don't see why this was done. In any case the implementation of Enum is extremely complicated and handles a lot of corner cases.

I realize this is not a cut-and-dried answer to your question but I hope it will point you to a solution.