How Kafka distributes the topic partitions among the brokers

Question

I have 3 Kafka brokers in 3 different VMs, with one additionally running a Zookeeper. I now create a topic with 8 partitions. The Producer pushes messages to these group of brokers on the created "topic".

• How does the Kafka distribute a topic and its partitions among the brokers?
• Does the Kafka redistribute the topic when a new Kafka Broker joins a cluster?
• Can the topic partition be increased after the topic was created?

Answer 1

• When you create a new topic, Kafka places the partitions and replicas in a way that the brokers with least number of existing partitions are used first, and replicas for same partition are on different brokers.

• When you add a new broker, it is used for new partitions (since it has lowest number of existing partitions), but there is no automatic balancing of existing partitions to the new broker. You can use the replica-reassignment tool to move partitions and replicas to the new broker.

• Yes, you can add partitions to an existing topic.