CODEWITHSUNDEEP
apache-kafka-streams
cddr
cddr

Reputation: 300

How can you tell when a Kafka Streams app is in the "running" state?

Given a recently started Kafka Streams app, how can one reliably determine that it has reached the "RUNNING" state? This is in the context of a test program that launches one or more streams apps and needs to wait until they are running before submitting test messages.

I know about the .setStateListener method but I'm wondering if there is a way of detecting this state from outside the app process. I thought it might be exposed as a jmx metric but I couldn't find one in VisualVM

Upvotes: 2

Views: 2567

Answers (1)

miguno
miguno

Reputation: 15067

The state listener method is the way to go. There is no other out-of-the-box way to achieve what you want.

That said, you can do the following for example:

  1. Expose a simple "health check" (or "running yes/no check") in your Kafka Streams application, e.g. via a REST endpoint (use whatever REST tooling you are familiar with).
  2. The health check can be based on Kafka Streams' built-in state listener, which you already know about.
  3. Your test program can then remotely query the health check endpoints of your various Kafka Streams application to determine when all of them are up and running.

Of course, you can use other ways to communicate readiness of a Kafka Streams application. The REST endpoint idea in (1) is just one example.

  • You can also let the Kafka Streams application write its readiness status into a Kafka topic, and your test program will subscribe to that topic to determine when all apps are ready.
  • Another option would be to provide a custom JMX metric in your Kafka Streams apps that your test program can then access.

Upvotes: 2

Related Questions