CODEWITHSUNDEEP
javaspring-bootapache-kafka-streams
Viktor Sarkozi
Viktor Sarkozi

Reputation: 23

Kafka through microservices, subscribe to non exists topics

I have a system build up multiple microservices (java spring boot), communicating via grpc and avro messages. It's using Kafka queue. My problem is: the services are building up in order, and one of my service subscribe to the topic of Kafka, before the other one sends a message and creates it. I do not get back any error, so it seems like working, but not. The topic doesnt exist at that time, and when it's getting messages, the consumer services doesn't notice it. If I restart client service it can join to topic and work normally. I would like to solve that the consumer could create that topic who is subscribed on, or try to reconnect if do not get any message.

Upvotes: 2

Views: 1170

Answers (1)

RyanWilcox
RyanWilcox

Reputation: 13972

Yes, I've seen this problem also. The solution that we came up with / the easiest one for this and other reasons, was to bake creating topics in with the deployment of the microservice. (Long story short: we call kafka-topics.sh on our CI/CD box, targetting the appropriate Zookeeper and brokers)

If your deployment scripts for the microservice know what topics to create, and make sure they are created before the application even starts, then problem avoided.

Upvotes: 1

Related Questions