CODEWITHSUNDEEP
javaapache-kafkaibm-cloudquarkus
Niklas Heidloff
Niklas Heidloff

Reputation: 962

How can you use TLS for Kafka in Quarkus?

The Kafka guide from Quarkus works nicely when running Kafka locally in Docker. I'm trying to change this sample by replacing the local Kafka service with a hosted Kafka service in the cloud which requires TLS.

Does anyone know how I can configure this? In the Quarkus documentation and the Smallrye documentation I don't see any properties for this.

I'd like to use the Kafka service in the IBM Cloud. Based on the documentation I've tried the following configuration in application.properties:

kafka.bootstrap.servers=broker-0-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-4-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-3-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-5-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-2-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-1-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093
kafka.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="token" password="...";
kafka.sasl.mechanism=PLAIN
kafka.security.protocol=SASL_SSL
kafka.ssl.protocol=TLSv1.2

Update: I've also tried Gunnar's suggestion below, but it doesn't work. When I use the following application.properties ...

mp.messaging.outgoing.generated-price.connector=smallrye-kafka
mp.messaging.outgoing.generated-price.topic=prices
mp.messaging.outgoing.generated-price.value.serializer=org.apache.kafka.common.serialization.IntegerSerializer

mp.messaging.outgoing.generated-price.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="token" password="...";
mp.messaging.outgoing.generated-price.sasl.mechanism=PLAIN
mp.messaging.outgoing.generated-price.security.protocol=SASL_SSL
mp.messaging.outgoing.generated-price.ssl.protocol=TLSv1.2

mp.messaging.incoming.prices.connector=smallrye-kafka
mp.messaging.incoming.prices.topic=prices
mp.messaging.incoming.prices.value.deserializer=org.apache.kafka.common.serialization.IntegerDeserializer

mp.messaging.outgoing.prices.sasl.jaas.config=org.apache.kafka.common.security.plain.PlainLoginModule required username="token" password="...";
mp.messaging.outgoing.prices.sasl.mechanism=PLAIN
mp.messaging.outgoing.prices.security.protocol=SASL_SSL
mp.messaging.outgoing.prices.ssl.protocol=TLSv1.2

kafka.bootstrap.servers=broker-0-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-4-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-3-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-5-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-2-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093,broker-1-8c8cph49mx2p2wqy.kafka.svc01.us-south.eventstreams.cloud.ibm.com:9093

... I get an error: javax.enterprise.inject.spi.DeploymentException: java.lang.IllegalArgumentException: Invalid channel configuration - the connector attribute must be set for channel prices at io.quarkus.smallrye.reactivemessaging.runtime.SmallRyeReactiveMessagingLifecycle.onApplicationStart(SmallRyeReactiveMessagingLifecycle.java:22)

Is TLS currently possible for Kafka in Quarkus?

Thanks

Upvotes: 6

Views: 4795

Answers (2)

ashutosh5111
ashutosh5111

Reputation: 11

One property is incorrect in the accepted answer by @Gunnar. It should be "security" instead of "ssl" in the property name.

mp.messaging.outgoing.generated-price.security.protocol=SSL

Upvotes: 0

Gunnar
Gunnar

Reputation: 18990

Have you tried specifying the relevant properties at the channel level? E.g. 

mp.messaging.outgoing.generated-price.connector=smallrye-kafka
mp.messaging.outgoing.generated-price.topic=mytopic
mp.messaging.outgoing.generated-price.ssl.protocol=...
mp.messaging.outgoing.generated-price.ssl.keystore.location=...
mp.messaging.outgoing.generated-price.ssl.keystore.password=...

You also could refer to variables when requiring the same values for multiple topics.

Upvotes: 4

Related Questions