Can kafka subscribe on a MQTT topic

Question

I have a MQTT broker running on a raspberry pi, i can publish and subscribe to topics.

But i would like it if Kafka could subscribe to one or multiple MQTT topics. Can somebody help me please?

Answer 1

yes you can do that.

• First of all you need to set up Kafka and zookeeper (here is the official apache kafka installation guide) (here is the official zookeeper installation guide)

• You need a kafka plugin to act as an MQTT source connector (you can use this one evokly/kafka-connect-mqtt )

• After compiling the source, you need to configure the plugin (indicate the MQTT topic, username, password, MQTT broker IP) (here is an example of the configuration)

• Restart kafka server and start the Kafka MQTT source connect

• start a kafka consumer that will subscibes to the kafka topic that you configured and retrieve messages (you can refer to this quick example) or you can develop your own consumer (here is an example how to develop your own kafka consumer Kafka Consumer in Java)