Mapping configuration to classes in python

Question

I am writing an application with complex configuration, handled through muliple layers (development, production, etc.) and by multiple sources (JSON files, environment variables, etc.). These two requirements are handled by most configuration libraries (e.g. dynaconf).

In order to decouple my application from the configuration library, I would like its values to be accessed through dependency injection/identifiers, akin to .NET's options pattern; i.e. I would like to implement something similar to the following:

#settings.toml

[default]
[default.section1]
a = "foo"
b = "bar"
[default.section2]
x = 1
y = 2

[dev]
[dev.section1]
a = "foodev"

[prod]
[prod.section2]
y = -2

#test.py

#provided by the configuration library, returns whole configuration as a flat dict with keys in the format "[section_name].[key]"
#settings = get_config("settings.toml", ...other sources...)

@Option("section1")
class Section1Options:
   a: str
   b: str

@Option("section2")
class Section2Options:
   x: int
   y: int

class MyService:
   def __init__(self, opts : Section2Options): #option injection
      self.x = opts .x
      self.z = opts .x + opts .y
      
      #equivalent to:
      #self.x = settings["section2.x"]
      #self.z = settings["section2.x"] + settings["section2.y"]

where @Option(section) returns a class decorator which sets any attribute of the class to a property returning the key within section which has the same name as attribute.

How would I go on about making such a decorator, and is it a good idea? Are there any libraries that already do this?

Implementation notes:

• I am considering having @Options behave similarly to @dataclass (i.e. extending it). This would allow to have a distinction between fields (attributes automatically mapped to configuration) and pseudo-fields (attributes which aren't).
• It would be useful to provide a way to define custom logic for mapping attributes to configuration keys (overriding the default lookup by the attribute's name).

Edit #1:

How do I make mapping work with configuration sections which have multiple levels like it does in .NET core? e.g.

{
    "hosts": {
        "servers": [
            {
                "hostname": "server1",
                "address": "10.106.10.32"
            },
            {
                "hostname": "server2",
                "address": "10.106.10.33"
            }
        ],
        "clients": [
            {
                "hostname": "foo",
                "address": "192.168.10.11"
            },
            {
                "hostname": "bar",
                "address": "192.168.10.12"
            },
            {
                "hostname": "baz",
                "address": "192.168.10.42"
            }
        ]
    }
}

would be mapped to the following classes:

@dataclass
class HostsOptions:
    servers : List[_HostOptions]
    clients : List[_HostOptions]

@dataclass
class _HostOptions:
    hostname : str
    address : str

Relevant:

.NET's ConfigBinder.Bind

"attempts to bind the given object instance to configuration values by matching property names against configuration keys recursively"

I have the feeling this recursive lookup process might have something in common with the concept of serialization

Answer 1

Implementing a decorator to map configuration sections to class attributes is possible and can improve code decoupling.

This is the code:

import functools
from dataclasses import dataclass
import json
import os


# Function to fetch the configuration (as an example, it loads from a JSON file)
def get_config():
    with open("settings.json", "r") as f:
        file_config = json.load(f)

    # Override with environment variables if necessary
    env_config = {k.lower().replace("section_", ""): v for k, v in os.environ.items() if k.startswith("SECTION")}
    
    # Combine file and environment configuration (env takes priority)
    combined_config = {**file_config, **env_config}
    
    return combined_config


# The Option decorator for injecting the configuration section into class attributes
def Option(section_name):
    def decorator(cls):
        cls = dataclass(cls)  # Convert class to a dataclass

        original_init = cls.__init__

        @functools.wraps(original_init)
        def __init__(self, *args, **kwargs):
            config = get_config()  # Get the full configuration
            section = config.get(section_name, {})  # Get the specific section

            # Set attributes based on the config section
            for field in cls.__dataclass_fields__:
                if field in section:
                    kwargs[field] = section[field]

            original_init(self, *args, **kwargs)

        cls.__init__ = __init__
        return cls
    return decorator


# Example configuration classes
@Option("section1")
class Section1Options:
    a: str
    b: str


@Option("section2")
class Section2Options:
    x: int
    y: int


# Example service class using dependency injection
class MyService:
    def __init__(self, opts: Section2Options):
        self.x = opts.x
        self.z = opts.x + opts.y

I hope this will help you a little.