CODEWITHSUNDEEP
javaspring-bootsslenvironment-variablesspring-kafka
Cameron Sours
Cameron Sours

Reputation: 380

Spring Boot/Spring Kafka SSL Configuration by environment variables impossible

I have a spring boot application which communicates with Kafka.

I configure this application in production by injecting environment variables.

For kafka, I can configure most things with environment variables - bootstrap servers, ssl truststore location, ssl truststore password, group id, topic, eg: 

SPRING_KAFKA_SSL_TRUSTSTORE-LOCATION: "file:/opt/app/jks/totally_real_file.jks"
SPRING_KAFKA_SSL_TRUSTSTORE-PASSWORD: "hunter2"

Because I can configure ssl keystore and password with environment variables, I would assume that I could configure the SSL Protocol and Security Protocol; eg:

SPRING_KAFKA_PROPERTIES_SECURITY_PROTOCOL: "SSL"
SPRING_KAFKA_PROPERTIES_SSL_PROTOCOL: "SSL"

But, I would assume incorrectly, because when I set it there, a message appears:

The configuration 'SECURITY_PROTOCOL' was supplied but isn't a known config
The configuration 'SSL_PROTOCOL' was supplied but isn't a known config

More messages appear!

Bootstrap broker one:9093 disconnected
Bootstrap broker two:9093 disconnected
Bootstrap broker more_than_two:9093 disconnected

This makes me sad. I search the internet. I find links:

Spring Kafka SSL setup in Spring boot application.yml

This doesn't help much, I want to configure by environment variable...

I find github issues links (this is getting worse):

https://github.com/spring-projects/spring-integration-kafka/issues/157

This gives a partial clue.

The yml:

spring.kafka.properties.security.protocol: "SSL"

should work, but doesn't

The yml:

spring:
  kafka:
    properties:
      security.protocol: "SSL"
      ssl.protocol: "SSL"

DOES work! but it is impossible to represent with environment variables.

Upvotes: 3

Views: 19604

Answers (3)

Bret Marzolf
Bret Marzolf

Reputation: 1

It may be that the environment variable names aren't correct. For example, instead of this:

SPRING_KAFKA_PROPERTIES_SECURITY_PROTOCOL

I think you want to use a producer/consumer-specific name like:

SPRING_KAFKA_PRODUCER_PROPERTIES_SECURITY_PROTOCOL

or

SPRING_KAFKA_CONSUMER_PROPERTIES_SECURITY_PROTOCOL

Upvotes: 0

Gary Russell
Gary Russell

Reputation: 174554

It's impossible for boot to consider every possible Kafka property. There are so many. Only a subset are supported as first class properties.

You can use system properties instead of environment variables.

EDIT

You can do it as follows:

spring:
  kafka:
    properties:
      security.protocol: ${SEC_PROT}

Upvotes: 3

Cameron Sours
Cameron Sours

Reputation: 380

My workaround was to set the following in my application.yml:

spring:
  kafka:
    properties:
      security.protocol: "SSL"
      ssl.protocol: "SSL"

I do not like this solution, as it requires me to build different artifacts for prod and SIT. (The fact that SIT Kafka does not have SSL enabled is a question for another day and another drink)

Upvotes: 1

Related Questions