Spring Boot: Override convention used to find application.properties config files

Question

I was looking at the spring-boot documentation located here

Specifically the section regarding the order in which the properties are considered:

More specifically:

Profile-specific application properties packaged inside your jar (application-{profile}.properties and YAML variants)

Let me first mention that I am not having any issues loading profile specific configurations using this approach(provided that the files are located in classpath:/ or classpath:/config.

However, what I am hoping to do is implement a convention like the following:

classpath:/default/application.properties
classpath:/{profile}/application.properties

Furthermore I'd like to achieve this configuration without making use of the spring.config.location property. I'm pretty new to Spring Boot so I'm looking for some hints as how to how I would implement this convention. Based on my research It seems that this might be achievable by adding a custom ConfigFileApplicationListener. Please let me know if that is a sensible starting point or any other ideas that might be better.

Update: It seems that if I could programmatically build out the spring.config.location list of properties I could pass in locations such as classpath:/default, classpath:{profile}. based on the spring.profiles.active environment variable. The following ConfigFileApplicationListener seems like its the one I want to call:

public void setSearchLocations(String locations)

However, I'm not sure where in the lifecycle I would make such a call.

Answer 1

A solution that doesn't require writing a new class:

public static void main(String[] args) {
    SpringApplication app = new SpringApplication();
    app.getListeners().stream()
            .filter(listener -> listener instanceof ConfigFileApplicationListener)
            .forEach(configListener -> {
                ((ConfigFileApplicationListener) configListener).setSearchLocations(mySearchLocations);
                ((ConfigFileApplicationListener) configListener).setSearchNames(mySearchNames);
            });
    app.setSources(singleton(MyClassName.class));
    app.run(args);
}

Answer 2

We did something similar with an EnvironmentPostProcessor to achieve the following naming convention:

1. System properties
2. environment variables
3. "random" (not used, but we kept that default PropertySource)
4. file:${foo.home}/foo-<profile>.properties
5. classpath*:<appName-profile>.properties
6. classpath*:application-profile.properties
7. classpath*:<appName>.properties
8. classpath*:application.properties
9. classpath*:meta.properties

Some applications do not have their own <appName>; those that do call setApplicationName in the main class's static initializer to use those two additional files.

The hacky part here is that we do not exclude the default ConfigFileApplicationListener, but undo it by removing PropertySource ConfigFileApplicationListener.APPLICATION_CONFIGURATION_PROPERTY_SOURCE_NAME.

File FooPropertiesEnvPostProcessor.java

package example.foo.utils.spring;

import static org.springframework.core.env.AbstractEnvironment.DEFAULT_PROFILES_PROPERTY_NAME;
import java.io.IOException;
import java.util.List;
import java.util.Spliterators;
import java.util.stream.Collectors;
import java.util.stream.Stream;
import java.util.stream.StreamSupport;
import org.springframework.beans.factory.annotation.AutowiredAnnotationBeanPostProcessor;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.context.config.ConfigFileApplicationListener;
import org.springframework.boot.env.EnvironmentPostProcessor;
import org.springframework.boot.env.PropertySourceLoader;
import org.springframework.boot.env.PropertySourcesLoader;
import org.springframework.boot.logging.LoggingApplicationListener;
import org.springframework.core.Ordered;
import org.springframework.core.env.AbstractEnvironment;
import org.springframework.core.env.ConfigurableEnvironment;
import org.springframework.core.env.MutablePropertySources;
import org.springframework.core.env.PropertyResolver;
import org.springframework.core.env.PropertySource;
import org.springframework.core.env.StandardEnvironment;
import org.springframework.core.io.Resource;
import org.springframework.core.io.ResourceLoader;
import org.springframework.core.io.support.PathMatchingResourcePatternResolver;
import org.springframework.core.io.support.ResourcePatternResolver;
import org.springframework.core.io.support.ResourcePatternUtils;
import org.springframework.core.io.support.SpringFactoriesLoader;

/**
 * Configures environment properties according to the FOO conventions.
 */
public class FooPropertiesEnvPostProcessor implements EnvironmentPostProcessor, Ordered {

    /**
     * Order before LoggingApplicationListener and before
     * AutowiredAnnotationBeanPostProcessor. The position relative to
     * ConfigFileApplicationListener (which we want to override) should not
     * matter: If it runs before this, we remove its PropertySource; otherwise,
     * its PropertySource remains but should do no harm as it is added at the
     * end.
     */
    public static final int ORDER
        = Math.min(LoggingApplicationListener.DEFAULT_ORDER, new AutowiredAnnotationBeanPostProcessor().getOrder()) - 1;

    static {
        System.setProperty(AbstractEnvironment.DEFAULT_PROFILES_PROPERTY_NAME,
                System.getProperty(AbstractEnvironment.DEFAULT_PROFILES_PROPERTY_NAME, "production"));
    }

    public FooPropertiesEnvPostProcessor() {
    }

    /**
     * Property key used as the application (sub-project) specific part in
     * properties file names.
     * <p>
     * <strong>Note:</strong> Direct access to this property key is meant for
     * tests which set the property in an annotation (e.g.
     * {@link IntegrationTest}). However, SpringBootApplications which need to
     * set this system property before Spring initialization should call
     * {@link #setApplicationName(String) setApplicationName} instead.
     * </p>
     */
    public static final String APP_KEY = "foo.config.name";

    /**
     * Sets the application name used to find property files (using
     * {@link FooPropertiesEnvPostProcessor}).
     *
     * @param appName
     *            the application name
     */
    public static void setApplicationName(String appName) {
        System.setProperty(APP_KEY, appName);
    }

    /**
     * Replacement for logging, which is not yet initialized during
     * postProcessEnvironment.
     */
    static void log(String format, Object... args) {
        System.out.println(String.format(format, args));
    }

    static void debug(PropertyResolver env, String format, Object... args) {
        String level = env.getProperty("logging.level." + FooPropertiesEnvPostProcessor.class.getName());
        if ("trace".equalsIgnoreCase(level) || "debug".equalsIgnoreCase(level)) {
            log(format, args);
        }
    }

    static void trace(PropertyResolver env, String format, Object... args) {
        String level = env.getProperty("logging.level." + FooPropertiesEnvPostProcessor.class.getName());
        if ("trace".equalsIgnoreCase(level)) {
            log(format, args);
        }
    }

    @Override
    public int getOrder() {
        return ORDER;
    }

    @Override
    public void postProcessEnvironment(ConfigurableEnvironment environment, SpringApplication application) {
        addProperties(environment.getPropertySources(), application.getResourceLoader(), environment);
    }

    public static void addProperties(MutablePropertySources propSources, ResourceLoader resLoader, ConfigurableEnvironment propRes) {
        trace(propRes, "FooPropertiesEnvPostProcessor.addProperties(..)");
        List<PropertySourceLoader> psls = SpringFactoriesLoader.loadFactories(PropertySourceLoader.class,
                PropertySourcesLoader.class.getClassLoader());
        // ResourcePatternUtils does not accept null yet
        // (https://jira.spring.io/browse/SPR-14500)
        ResourcePatternResolver rpr = resLoader != null ? ResourcePatternUtils.getResourcePatternResolver(resLoader)
                : new PathMatchingResourcePatternResolver();
        final String suffix = ".properties"; // SonarQube made me declare this
        String[] profiles = propRes.getActiveProfiles();
        if (profiles.length == 0) {
            profiles = new String[] { System.getProperty(DEFAULT_PROFILES_PROPERTY_NAME) };
        }

        // ConfigFileApplicationListener adds PropertySource "applicationConfigurationProperties" consisting of
        // - "applicationConfig: [classpath:/${spring.config.name}-<profile>.properties]"
        // - "applicationConfig: [classpath:/${spring.config.name}.properties]"
        // Since we want the profile to have higher priority than the app name, we cannot just set
        // "spring.config.name" to the app name, use ConfigFileApplicationListener, and add
        // "application-<profile>.properties" and "application.properties".
        // Instead, remove ConfigFileApplicationListener:
        PropertySource<?> removedPropSource = propSources.remove(ConfigFileApplicationListener.APPLICATION_CONFIGURATION_PROPERTY_SOURCE_NAME);
        trace(propRes, "removed %s from %s", removedPropSource, propSources);

        // add meta.properties at last position, then others before the previously added. => resulting order:
        // - { systemProperties
        // - systemEnvironment
        // - random } - already added automatically elsewhere
        // - file:${foo.home}/foo-<profile>.properties
        // - classpath:<appName>-<profile>.properties
        // - classpath:application-<profile>.properties
        // - classpath:<appName>.properties
        // - classpath:application.properties
        // - classpath:meta.properties
        // By adding ${foo.home}/... (chronlogically) last, the property can be set in the previously added resources.
        boolean defaultAppName = "application".equals(propRes.resolveRequiredPlaceholders("${" + APP_KEY + ":application}"));
        String psn = null;
        psn = addProperties(propSources, propRes, rpr, psls, true, psn, propRes.resolveRequiredPlaceholders("classpath*:meta" + suffix));
        psn = addProperties(propSources, propRes, rpr, psls, true, psn, propRes.resolveRequiredPlaceholders("classpath*:application" + suffix));
        if (!defaultAppName) {
            psn = addProperties(propSources, propRes, rpr, psls, false,
                    psn, propRes.resolveRequiredPlaceholders("classpath*:${" + APP_KEY + ":application}" + suffix));
        }
        for (String profile : profiles) {
            psn = addProperties(propSources, propRes, rpr, psls, false, psn,
                    propRes.resolveRequiredPlaceholders("classpath*:application-" + profile + suffix));
        }
        if (!defaultAppName) {
            for (String profile : profiles) {
                psn = addProperties(propSources, propRes, rpr, psls, false,
                        psn, propRes.resolveRequiredPlaceholders("classpath*:${" + APP_KEY + ":application}-" + profile + suffix));
            }
        }
        for (String profile : profiles) {
            psn = addProperties(propSources, propRes, rpr, psls, false,
                    psn, propRes.resolveRequiredPlaceholders("file:${foo.home:.}/foo-" + profile + suffix));
        }

        Stream<PropertySource<?>> propSourcesStream = StreamSupport.stream(Spliterators.spliteratorUnknownSize(propSources.iterator(), 0), false);
        debug(propRes, "Property sources: %s%n", propSourcesStream.map(PropertySource::getName).collect(Collectors.joining(", ")));
    }

    /**
     * Adds a resource given by location string to the given PropertySources, if
     * it exists.
     *
     * @param propSources
     *            the property sources to modify
     * @param successorName
     *            the name of the (already added) successor resource, i.e. the
     *            resource before which the new one should be added; if null,
     *            add as last resource
     * @param location
     *            the location of the resource to add
     * @return the name of the newly added resource, or {@code successorName} if
     *         not added
     */
    private static String addProperties(MutablePropertySources propSources, PropertyResolver propRes, ResourcePatternResolver resLoader,
            List<PropertySourceLoader> propLoaders, boolean required, String successorName, String location) {
        Resource[] resources;
        try {
            resources = resLoader.getResources(location);
        } catch (IOException e) {
            throw new IllegalStateException("failed to load property source " + location + ": " + e, e);
        }
        if (resources.length == 0) {
            debug(propRes, "%s property resource not found: %s", required ? "required" : "optional", location);
            if (required) {
                throw new IllegalStateException("required property source " + location + " not found");
            } else {
                return successorName;
            }
        }

        String newSuccessorName = successorName;
        for (Resource resource : resources) {
            boolean exists = resource.exists();
            debug(propRes, "%s property resource %sfound: %s%s", required ? "required" : "optional", exists ? "" : "not ", location,
                    uriDescription(resource, propRes));
            if (!required && !exists) {
                continue;
            }

            boolean loaded = false;
            for (PropertySourceLoader propLoader : propLoaders) {
                if (canLoadFileExtension(propLoader, resource)) {
                    newSuccessorName = addResource(propSources, propRes, resource, propLoader, newSuccessorName);
                    loaded = true;
                    break;
                }
            }
            if (!loaded && required) {
                throw new IllegalStateException("No PropertySourceLoader found to load " + resource);
            }
        }
        return newSuccessorName;
    }

    private static String addResource(MutablePropertySources propSources, PropertyResolver propRes, Resource resource,
            PropertySourceLoader propLoader, String successorName) {
        try {
            PropertySource<?> propSource = propLoader.load(resource.getDescription(), resource, null);
            if (propSource == null) {
                // e.g. a properties file with everything commented;
                // org.springframework.boot.env.PropertiesPropertySourceLoader
                // converts empty to null
                return successorName;
            }
            if (successorName == null) {
                propSources.addLast(propSource);
            } else if (successorName.equals(propSource.getName())) {
                // happens if APP_KEY is not set, so that
                // "${APP_KEY:application}" == "application"
                trace(propRes, "skipping duplicate resource %s", successorName);
            } else {
                propSources.addBefore(successorName, propSource);
            }
            return propSource.getName();
        } catch (IOException e) {
            throw new IllegalStateException("Unable to load configuration file " + resource + ": " + e, e);
        }
    }

    /**
     * Stolen from {@link PropertySourcesLoader}
     */
    private static boolean canLoadFileExtension(PropertySourceLoader loader, Resource resource) {
        String filename = resource.getFilename().toLowerCase();
        for (String extension : loader.getFileExtensions()) {
            if (filename.endsWith("." + extension.toLowerCase())) {
                return true;
            }
        }
        return false;
    }

    private static String uriDescription(Resource resource, PropertyResolver propRes) {
        try {
            return resource.exists() ? (" in " + resource.getURI()) : "";
        } catch (IOException e) {
            trace(propRes, "getURI: %s", e);
            return "";
        }
    }
}

File META-INF/spring.factories

org.springframework.boot.env.EnvironmentPostProcessor = example.foo.utils.spring.FooPropertiesEnvPostProcessor

To get the same properties in tests, they have @ContextConfiguration(..., initializers = TestAppContextInitializer.class). TestAppContextInitializer implements ApplicationContextInitializer<GenericApplicationContext> and calls FooPropertiesEnvPostProcessor.addProperties in its initialize method.

Unfortunately, the EnvironmentPostProcessor seems to be missing Spring Shell by default, too. In our case (since only a tiny part of the application uses Spring Shell), it was sufficient to restrict the <context:component-scan base-package=.../> scope in META-INF/spring/spring-shell-plugin.xml to contain only stuff which does not need any properties set by the EnvironmentPostProcessor.

Answer 3

So here is what I managed to come up with, not sure if I'll even go with this solution but I figured I'll offer it up in case there is any helpful feedback.

So I resorted to trying to set the call the setSearchLocations(String locations) method on the ConfigFileApplicationListener after it has been added to the SpringApplication but before its triggered. I did this by adding a new listener that also implements Ordered and made sure it ran before ConfigFileApplicationListener. This seems to do what I want but I'm still thinking there is a more elegant approach. I especially dont like having to iterate over the Listeners.

public class LocationsSettingConfigFileApplicationListener implements
        ApplicationListener<ApplicationEnvironmentPreparedEvent>, Ordered {

    /**
     * this should run before ConfigFileApplicationListener so it can set its
     * state accordingly
     */
    @Override
    public int getOrder() {
        return ConfigFileApplicationListener.DEFAULT_ORDER - 1;
    }

    @Override
    public void onApplicationEvent(ApplicationEnvironmentPreparedEvent event) {

        SpringApplication app = event.getSpringApplication();
        ConfigurableEnvironment env = event.getEnvironment();

        for (ApplicationListener<?> listener : app.getListeners()) {

            if (listener instanceof ConfigFileApplicationListener) {
                ConfigFileApplicationListener cfal = (ConfigFileApplicationListener) listener;
                //getSearchLocations omitted
                cfal.setSearchLocations(getSearchLocations(env));
            }
        }

    }