CODEWITHSUNDEEP
apache-kafkaapache-zookeeper
jessica
jessica

Reputation: 2610

how to capture the retention bytes values from zookeeper meta data

we are trying to capture the retention bytes values from the topic - topic_test

we try the following example , but seems this isnt the right path from zookeeper

zookeeper-shell kafka1:2181,kafka2:2181,kafka3:2181 <<< "ls /brokers/topics/topic_test/partitions/88/state"
Connecting to kafka1:2181,kafka2:2181,kafka3:2181
Welcome to ZooKeeper!
JLine support is disabled

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
[]

any idea where are the values of retention bytes per topic that can capture from zookeeper?

I did the following , but not see the retention bytes ( what is wrong here ) , we have kafka confluent version - 0.1

zookeeper-shell confluent1:2181 get /config/topics/test_topic
Connecting to kafka1:2181

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
{"version":1,"config":{}}
cZxid = 0xb30a00000038
ctime = Mon Jun 29 11:42:30 GMT 2020
mZxid = 0xb30a00000038
mtime = Mon Jun 29 11:42:30 GMT 2020
pZxid = 0xb30a00000038
cversion = 0
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 25
numChildren = 0

Upvotes: 0

Views: 314

Answers (1)

Mickael Maison
Mickael Maison

Reputation: 26885

Configurations are stored in Zookeeper under the /config path.

For example, for the topic topic_test:

# Create topic
./bin/kafka-topics.sh --bootstrap-server localhost:9092 --create \
    --topic topic_test --partitions 1 --replication-factor 1 --config retention.bytes=12345

# Retrieve configs from Zookeeper
./bin/zookeeper-shell.sh localhost get /config/topics/topic_test
Connecting to localhost

WATCHER::

WatchedEvent state:SyncConnected type:None path:null
{"version":1,"config":{"retention.bytes":"12345"}}

Note in most cases, you should not rely on direct access to Zookeeper but instead use the Kafka API to retrieve these values.

Using:

  • kafka-topics.sh:

    ./bin/kafka-topics.sh --bootstrap-server localhost:9092 --describe --topic topic_test

  • kafka-configs.sh:

    ./bin/kafka-configs.sh --bootstrap-server localhost:9092 --describe --entity-type topics --entity-name topic_test

  • The Admin API using describeConfigs()

Upvotes: 2

Related Questions