Environment variables for list in spring boot configuration

Question

For my Spring Boot application, I am trying to use an environment variable that holds the list of properties.topics in application.yml (see configuration below).

properties:
      topics:
        - topic-01
        - topic-02
        - topic-03

I use the configuration file to populate properties bean (see this spring documentation), as shown below

import java.util.ArrayList;
import java.util.List;
import org.springframework.boot.context.properties.ConfigurationProperties;

@ConfigurationProperties("properties")
public class ApplicationProperties {
  private List<String> topics = new ArrayList<>();
  public void setTopics(List<String> topics) {
     this.topics = topics;
  }
  public List<String> getTopics() {
     return this.topics;
  }
}

With the use of environment variable, I can have the list's content change without changing the application.yml. However, all examples that I could find so far only for cases where an environment variable holding only single value, not a collection of values in my case.

Edit:

To make it clear after @vancleff's comment, I do not need the values of the environment variable to be saved to application.yml.

Another edit:

I think by oversimplifying my question, I shoot myself in the foot. @LppEdd answer works well with the example given in my question. However, what happens if instead of a collection of simple string topic names, I need a bit more complex structure. For example, something like

properties:
  topics:
    - 
      name: topic-01
      id: id-1
    - 
      name: topic-02
      id: id-2
    - 
      name: topic-03
      id: id-3

Answer 1

I fixed this with having a array in deployment.yaml from values.yml replacing the default values of application.yml

Example as:

deployment.yml -

----------------

env: 
            - name : SUBSCRIBTION_SITES_0_DATAPROVIDER
              value: {{ (index .Values.subscription.sites 0).dataprovider | quote }}
            - name: SUBSCRIBTION_SITES_0_NAME
              value: {{ (index .Values.subscription.sites 0).name | quote }}

values.yml -

---------------

  subscription:
    sites:
      - dataprovider: abc
        name: static

application.yml -

------------------

  subscription:
    sites:
      - dataprovider: ${SUBSCRIBTION_SITES_0_DATAPROVIDER:abcd}
        name: ${SUBSCRIBTION_SITES_0_NAME:static}
    

Java Code :

@Getter
@Setter
@Configuration
@ConfigurationProperties(prefix = "subscription")
public class DetailsProperties {

    private List<DetailsDto> sites;

    DetailsProperties() {
        this.sites = new ArrayList<>();
    }

}

Pojo mapped :

@Getter
@Setter
@Data
@Builder
@AllArgsConstructor
@NoArgsConstructor
public class DetailsDto {
    private String dataprovider;
    private String name;
}

Answer 2

Yet another way to solve this, it is uglier in the parsing but very easy and works. Replace:

properties:
  topics:
    - 
      name: topic-01
      id: id-1
    - 
      name: topic-02
      id: id-2
    - 
      name: topic-03
      id: id-3

With:

properties:
  topics: topic-01,topic-02,topic-03
  id: id-1,id-2,id-3

And read it like this:

private List<CombinedTopic> topicsCombined = new ArrayList<>();
public CombinedTopic(@Value("${properties.topics}") List<String> topics,
                     @Value("${properties.id}") List<String> ids) {
    List<CombinedTopic> collect = IntStream.range(0, ids.size())
        .mapToObj(i -> new CombinedTopic(ids.get(i), topics.get(i)))
            .toList();
}

Note that I assume that ID and Topic must go together and in the same order.

Answer 3

I built a quick little utility to do this.

import java.util.LinkedList;
import java.util.List;

import org.springframework.core.env.Environment;

/**
 * Convenience methods for dealing with properties.
 */
public final class PropertyUtils {

  private PropertyUtils() {
  }

  public static List<String> getPropertyArray(Environment environment, String propertyName) {
    final List<String> arrayPropertyAsList = new LinkedList<>();
    int i = 0;
    String value;
    do {
      value = environment.getProperty(propertyName + "[" + i++ + "]");
      if (value != null) {
        arrayPropertyAsList.add(value);
      }
    } while (value != null);

    return arrayPropertyAsList;
  }

}

You could modify this without too many changes to support multiple fields as well. I've seen similar things done to load an array of database configurations from properties.

Answer 4

a bit late for the show but, I was facing the same problem and this solves it

https://github.com/spring-projects/spring-boot/wiki/Relaxed-Binding-2.0#lists-1

MY_FOO_1_ = my.foo[1]

MY_FOO_1_BAR = my.foo[1].bar

MY_FOO_1_2_ = my.foo[1][2]`

So, for the example in the question:

properties:
  topics:
    - 
      name: topic-01
      id: id-1
    - 
      name: topic-02
      id: id-2
    - 
      name: topic-03
      id: id-3

The environment variables should look like this:

PROPERTIES_TOPICS_0_NAME=topic-01
PROPERTIES_TOPICS_0_ID=id-01
PROPERTIES_TOPICS_1_NAME=topic-02
PROPERTIES_TOPICS_1_ID=id-02
PROPERTIES_TOPICS_2_NAME=topic-03
PROPERTIES_TOPICS_2_ID=id-03

Answer 5

Suggestion, don't overcomplicate.

Say you want that list as an Environment variable. You'd set it using

-Dtopics=topic-01,topic-02,topic-03

You then can recover it using the injected Environment Bean, and create a new List<String> Bean

@Bean
@Qualifier("topics")
List<String> topics(final Environment environment) {
    final var topics = environment.getProperty("topics", "");
    return Arrays.asList(topics.split(","));
}

From now on, that List can be @Autowired.
You can also consider creating your custom qualifier annotation, maybe @Topics.

Then

@Service
class TopicService {
   @Topics
   @Autowired
   private List<String> topics;

   ...
}

Or even

@Service
class TopicService {
   private final List<String> topics;

   TopicService(@Topics final List<String> topics) {
      this.topics = topics;
   }

   ...
}

---

What you could do is use an externalized file.
Pass to the environment parameters the path to that file.

-DtopicsPath=C:/whatever/path/file.json

Than use the Environment Bean to recover that path. Read the file content and ask Jackson to deserialize it

You'd also need to create a simple Topic class

public class Topic {
    public String name;
    public String id;
}

Which represents an element of this JSON array

[
    {
        "name": "topic-1",
        "id": "id-1"
    },
    {
        "name": "topic-2",
        "id": "id-2"
    }
]

---

@Bean
List<Topic> topics(
        final Environment environment,
        final ObjectMapper objectMapper) throws IOException {
    // Get the file path
    final var topicsPath = environment.getProperty("topicsPath");

    if (topicsPath == null) {
        return Collections.emptyList();
    }

    // Read the file content
    final var json = Files.readString(Paths.get(topicsPath));

    // Convert the JSON to Java objects
    final var topics = objectMapper.readValue(json, Topic[].class);
    return Arrays.asList(topics);
}

---