CODEWITHSUNDEEP
apache-kafka
gillyb
gillyb

Reputation: 8910

Where kafka stores partitions for the topics?

I installed kafka on a linux server. I defined a topic with a few partitions. I know that each partition is mapped to a physical file on disk, but I don't know where it is.
Where are the partition files saved ?

Upvotes: 8

Views: 18102

Answers (4)

Pankaj Kumar
Pankaj Kumar

Reputation: 300

I have started Kafka standalone server with default properties in Windows and found the Topic and partitions details in the below directory

C:\tmp\kafka-logs

enter image description here

Upvotes: 0

Mouse
Mouse

Reputation: 11

Try running this command 

bin/kafka-topics.sh --zookeeper localhost:2181 --describe --topic test

you will get output

Topic:test  Partition: 0    Leader: 1   Replicas: 1,2,0 Isr: 1,2,0

now try going to \config file

cat server.properties

and search for broker_id

if broker_id matches with leader number then topic partition is stored in that broker

Upvotes: 0

Salvador Dali
Salvador Dali

Reputation: 222761

By default on Linux it is stored in /tmp/kafka-logs. If you will navigate to this folder you will see something like this:

  • recovery-point-offset-checkpoint
  • replication-offset-checkpoint
  • topic-0
  • msg-0
  • msg-1

Which means that you have two topics (topic which has 1 partition and msg which has 2).

As it was noted by Ludd, you can find the location inside config/server.properties file by looking for log.dirs.

Upvotes: 7

Lundahl
Lundahl

Reputation: 6562

In your config/server.properties you'll find a section on "Log Basics". The property log.dirs is defining where your logs/partitions will be stored on disk.

Upvotes: 19

Related Questions