CODEWITHSUNDEEP
apache-kafkapartitionapache-zookeeper
fcbflying
fcbflying

Reputation: 703

kafka different topics set different partitions

As I know, 'num.partitions' in kafka server.properties will be worked for all topics. Now I want to set partitionNumber=1 for topicA and partitionNumber=2 for topicB. Is that possible to implementation with high level api?

Upvotes: 0

Views: 193

Answers (2)

Morgan Kenyon
Morgan Kenyon

Reputation: 3172

There a configuration value that can be set on a Kafka Broker.

auto.create.topics.enable=true

True is actually the default setting,

Enable auto creation of topic on the server. If this is set to true then attempts to produce data or fetch metadata for a non-existent topic will automatically create it with the default replication factor and number of partitions.

So if you read or write from a non-existent partition as if it existed, if will automatically create one for you. I've never heard of using the high level api to automatically create one.

Looking over the Kafka Protocol Documentation, there doesn't seem to be a provided way to create topics.

Upvotes: 0

Heejin
Heejin

Reputation: 4571

num.partitions is a value used when a topic is generated automatically. If you generate a topic yourself, you can set any number of partitions as you want.

You can generate a topic yourself with the following command. (replication factor 3 and the number of partitions 2. Capital words are what you have to replace.)

bin/kafka-topics.sh --create --zookeeper ZOOKEEPER_HOSTNAME:ZOOKEEPER_PORT \
--replication-factor 3 --partitions 2 --topic TOPIC_NAME

Upvotes: 2

Related Questions