Srw
Srw

Reputation: 11

Kafka streams with multiple application instances

I have 2 instances of my application for kafka streams consuming 2 partitions in a single topic.

will the single partitions data be only in one application or both applications? and Say if one applications instance is down will i have issues. how will interactive queries solve this ? do i need to use globalktable?

Upvotes: 0

Views: 2098

Answers (1)

Nishu Tayal
Nishu Tayal

Reputation: 20830

Each kafka stream application instance will be mapped to one or more partition, based on how many partitions the input topics have.

If you run 2 instances for an input topic with 2 partitions, each partition will consume from one partition. If one instance goes down, kafka stream will rebalance the work load on the first instance and it will consumer from both partition.

You can refer the architecture here in detail : https://docs.confluent.io/current/streams/architecture.html enter image description here

Upvotes: 1

Related Questions