CODEWITHSUNDEEP
apache-kafka
Apostolos Lazidis
Apostolos Lazidis

Reputation: 145

Scale-up Kafka without downtime

Lets say that we have a service A,B and a Kafka Broker in different VMs. Service_A is the producer, Service_B is the consumer.

If Kafka can't handle workload then:

a) How can it scale-up without downtime?

Also,

b) Can number of partitions change in real-time without downtime?

Upvotes: 0

Views: 405

Answers (1)

OneCricketeer
OneCricketeer

Reputation: 191671

The answer to both is yes, but scaling out Kafka (adding brokers) will not add partitions to existing topics; you'll need to manually invoke kafka-reassign-partitions command, which requires brokers to be running.

The main problem you may run into with that is if there are clients actively using some partition on a broker and you completely move it off to a different broker.

You cannot reduce partitions, ever.

Upvotes: 1

Related Questions