CODEWITHSUNDEEP
apache-kafka
alexanoid
alexanoid

Reputation: 25770

Kafka log-compacted topic partition and disk space

I'm unable to find the information on how Kafka will handle the situation when the log-compacted topic partition will run out of space on the physical disk where it is located.

Will Kafka broker(and the data at all) - survive or fail in this case?

Upvotes: 0

Views: 386

Answers (1)

OneCricketeer
OneCricketeer

Reputation: 191681

Like any other OS process, Kafka will say it's unable to open a file because it's out of space.

The Kafka broker and other OS processes will begin to stop and/or error out.

You need monitoring on your disk volumes for at least the 80% threshold depending on the rate of increase. If you reach above 90%, you'll likely need to take the broker offline to add more storage or add more brokers and reassign the some partitions yourself

Some monitoring tools even allow you to use linear regression to predict when you'll reach capacity.

Upvotes: 1

Related Questions