Python 3.6 Referencing Enum Flag from config file

Question

Using Python 3.6 & Enum Flag (note: Enum is new to v3.4, Flag is new in v3.6)

I'm trying to figure out if there is a "automagic" way of assignment of Enum Flags that referenced dictionary by a string. In my case, a JSON config file is loaded with text specifying the Enum Flags but I have to perform a bunch of "if" statements to reassign the node to the actual Enum.

The example below works but gets kludgy when you have a a lot of references to udpate.

The config file is larger and contains text, numeric, and enums.

Sample Code

#import json
from enum import Enum, Flag

class stream_data(Flag):
    wind=  (0x00000001)
    humidity = (0x00000002)

#with open('strings.json') as json_file:
#    config = json.load(json_file)
# Assume we loaded with json.load
config = {
    "options": {
        "stream": {
            "capture_rate": "1", "stream_data": "wind", "stream_pull_rate": 5, "stream_type": "binary"
        }
    }
}
print('type before: %s' % type(config['options']['stream']['stream_data']))

if config['options']['stream']['stream_data'] == stream_data.wind.name:
        config['options']['stream']['stream_data'] = stream_data.wind

print('type after: %s' % type(config['options']['stream']['stream_data']))

results:

type before: <class 'str'>
type after: <enum 'stream_data'>

Is there some python magic for this that I am unware of?

(I was thinking I could iterate through the dict and enum classes to see if the names match but that also seem a bit kludgy.)

Answer 1

Enums expose their internal dict so you can access their values by name using:

stream_data["wind"]  # 1
stream_data["humidity"]  # 2
# etc.

Therefore, with your config (as c to fit without scrolling) set up you can do an automatic replacement as:

c['options']['stream']['stream_data'] = stream_data[c['options']['stream']['stream_data']]

UPDATE: If you want to automate everything you can write a small recursive function to go through your structure and attempt to replace values for all keys that match a particular Enum name from the global scope, for example:

import collections
import enum

def decode_enums(target, scope=None):  # a simple recursive enum value replacer
    if isinstance(target, collections.MutableMapping):  # check if dict-like...
        scope = scope or globals()  # if a scope dict is not passed, use globals()
        for k, v in target.items():  # iterate over all the items in the passed dictionary
            if k in scope and issubclass(scope[k], enum.Enum):  # enum found in the scope
                try:
                    target[k] = scope[k][v]  # replace with the 'enum' value
                    continue
                except KeyError:  # the value is not enumerated, fall-back to decoding
                    pass
            target[k] = decode_enums(v, scope)  # dig deeper...
    elif hasattr(target, "__iter__") and not isinstance(target, (str, bytes, bytearray)):
        for v in target:  # iterate over the elements of this iterable
            decode_enums(v)  # try to decode their values
    return target  # return the passed value to enable recursive assignment

Now you can define your enums to your heart content as:

import enum

class stream_data(enum.Flag):
    wind = 0x00000001
    humidity = 0x00000002

class stream_type(enum.Flag):  # just for the kick of it
    binary = 0x00000001
    text = 0x00000002

And then have your structure updated automagically with the actual Enum values:

def value_info(prefix, name, value):  # a small function to trace the value info
    type_ = type(value)
    print(f'{prefix:<6} - {name}: {value:>30}')
    print(f'{prefix:<6} - type({name}): {type_!s:>24}')

config = {
    "options": {
        "stream": {
            "capture_rate": "1", "stream_data": "wind",
            "stream_pull_rate": 5, "stream_type": "binary"
        }
    }
}

value_info("before", "stream_data", config["options"]["stream"]["stream_data"])
value_info("before", "stream_type", config["options"]["stream"]["stream_type"])

decode_enums(config)

value_info("after", "stream_data", config["options"]["stream"]["stream_data"])
value_info("after", "stream_type", config["options"]["stream"]["stream_type"])

Which will give you:

before - stream_data:                           wind
before - type(stream_data):            <class 'str'>
before - stream_type:                         binary
before - type(stream_type):            <class 'str'>
after  - stream_data:               stream_data.wind
after  - type(stream_data):     <enum 'stream_data'>
after  - stream_type:             stream_type.binary
after  - type(stream_type):     <enum 'stream_type'>

You can even have it not pick up the scope from globals() by defining a specific map for your enums:

class StreamData(enum.Flag):
    wind = 0x00000001
    humidity = 0x00000002

value_info("before", "stream_data", config["options"]["stream"]["stream_data"])
value_info("before", "stream_type", config["options"]["stream"]["stream_type"])

decode_enums(config, {"stream_data": StreamData})

value_info("after", "stream_data", config["options"]["stream"]["stream_data"])
value_info("after", "stream_type", config["options"]["stream"]["stream_type"])

Which will yield:

before - stream_data:                           wind
before - type(stream_data):            <class 'str'>
before - stream_type:                         binary
before - type(stream_type):            <class 'str'>
after  - stream_data:                StreamData.wind
after  - type(stream_data):      <enum 'StreamData'>
after  - stream_type:                         binary
after  - type(stream_type):            <class 'str'>

You can also create a counterpart encode_enums() function to use when storing your JSON that will do the same recursive walk but instead of linking the enums with their values it would turn back the values into names.

All this being said, given that you're losing important type info I'd recommend you to move to the YAML format instead. It allows you to create type extensions so you get to keep the enum metadata in your YAML structure and you don't have to traverse the whole structure after parsing just to try to guess the enum values.