When/how does a topic "marked for deletion" get finally removed?

Question

I have issued the command to delete a topic:

./bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic  vip_ips_alerts

It seemed to give a happy response:

[2014-05-31 20:58:10,112] INFO zookeeper state changed (SyncConnected) (org.I0Itec.zkclient.ZkClient)
Topic "vip_ips_alerts" queued for deletion.

But now 10 minutes later the topic still appears in the --list command:

./bin/kafka-topics.sh --zookeeper localhost:2181 --list
vip_ips_alerts - marked for deletion

So what does that mean? When will the topic be really deleted? How do I expedite this process?

Answer 1

2021/22 - rmr is deprecated, use deleteall

1. Login to Zookeeper:

Linux

Go to bin folder:

.\zookeeper-shell.sh localhost:2181

Windows

Go to bin\windows folder:

.\zookeeper-shell.bat localhost:2181

2. Delete topic

deleteall /brokers/topics/NameOfTopicToDelete

No need to run anymore: deleteall /admin/delete_topics/NameOfTopicToDelete

Restart zookeeper and kafka. If you try to delete topic now you will get:

Node does not exist error.

Answer 2

If nothing worked above , You can try below :

First, log into the Zookeeper CLI console using the proper ‘zkCli.sh’ file, you’ll find this in the ‘bin’ directory in your Zookeeper installation.

2. get /brokers/topics/<topic_name>

3. rmr /brokers/topics/<topic_name>

4. rmr /admin/delete_topics/<topic_name>

Answer 3

In Kafka, once the topic is marked for deletion, it will get finally removed after 60000 milliseconds.

Check the property log.segment.delete.delay.ms.

The documentation says:

file.delete.delay.ms: The time to wait before deleting a file from the filesystem Default value is 60000 ms.

Answer 4

Check status of topic at zookeeper

bin\windows>kafka-topics.bat --list --zookeeper localhost:2181
Output: topic shows marked for deletion

Setting delete.topic.enable=true in server.properties was not working as well.

Solution: Check the zookeeper data directory location in zookeeper.properties file. it was dataDir=/tmp/zookeeper.
Issue got resolved after updating "dataDir" to a new location.

dataDir=zk-temp
# the port at which the clients will connect
clientPort=2181
# disable the per-ip limit on the number of connections since this is a non-production config
maxClientCnxns=0

Answer 5

The deletion happened for me almost immediately (v 0.9). I believe that it should be the same for you.

Once marked, the deletion will be triggered on the kafka node which is the topic partition leader. The thing to remember is that the the topic partition leader must be configured correctly (in terms of listeners), otherwise it will not receive the instruction to delete the logs.

I had my "marked for deletion" topics stuck at that state until I corrected my server properties and restarted the respective node.

Answer 6

For kafka version 0.10.0.0 it is enough to enable the topic deletion by setting:

delete.topic.enable

The topic is deleted within few moments

kafka-topics --delete --zookeeper your-zk:2181 --topic yourTopicName

You can confirm that it is gone with the following command:

kafka-topics --describe --zookeeper your-zk:2181 --topic yourTopicName

Answer 7

We had this issue when deleting topics. The topics had been created when delete.topic.enable=true had not been set. We set that in config, restarted kafka to apply the new config. Deleted the topics and saw the "marked for deletion". We then restarted kafka again. After 40 minutes though all topics had been deleted (9 topics with total partitions in the thousands). The topics with higher numbers of partitions seemed to take longer, which initially made it look like nothing was happening.

Answer 8

I faced the same issue and spend a couple of days trying to identify with the issue was. I triggered command to delete the topics, however the topics were marked for deletion but they were not deleted.

1. I first checked the configuration which are set right. Under: server.properties

   delete.topic.enable=true for all brokers

2. I restarted the broker to check if the topics got delete (No!!).
3. I check no data under /kafka/data folder.
4. I even considered the option to wait until the retention time got exceed.

None helped. I had to finally login to zooker

./zkCli.sh # and delete the topics using 
rmr /brokers/topics/<<topic>> and rmr /admin/delete_topics/<<topic>>

Please remember to restart kafka after this. Hope this solves your issue.

Answer 9

My issue was something similar. I deleted a topic and it was giving me the same message when I was listing all the topics.

I brought down the zookeeper and the broker. set the delete.topic.enable=true in my broker config file started zookeeper and the boker

The topic was gone...thanks to Jacek Laskowski

Answer 10

You can do it.

sudo ./zookeeper-shell.sh localhost:2181 rmr /brokers/topics/your_topic

Answer 11

In my case where i am using Kafka 8.2.2, I had to delete entries from the following manually -

1. Delete topic folder from Kafka broker machine.

2. Login to zookeeper and -

   hbase zkcli
   rmr /brokers/topics/{topic_name}
   rmr /admin/delete_topics/{topic_name}
   

Answer 12

tl;dr Set delete.topic.enable = true in config/server.properties of Kafka brokers and...be patient.

It happens with the latest development version of Kafka 0.8.3-SNAPSHOT:

➜  kafka_2.11-0.8.3-SNAPSHOT git:(trunk) ✗ ./bin/kafka-topics.sh --zookeeper localhost:2181 --create --topic my-topic --partitions 2 --replication-factor 1
Created topic "my-topic".

➜  kafka_2.11-0.8.3-SNAPSHOT git:(trunk) ✗ ./bin/kafka-topics.sh --zookeeper localhost:2181 --describe --topic my-topic
Topic:my-topic  PartitionCount:2    ReplicationFactor:1 Configs:
    Topic: my-topic Partition: 0    Leader: 0   Replicas: 0 Isr: 0
    Topic: my-topic Partition: 1    Leader: 0   Replicas: 0 Isr: 0

➜  kafka_2.11-0.8.3-SNAPSHOT git:(trunk) ✗ ./bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic my-topic
Topic my-topic is marked for deletion.
Note: This will have no impact if delete.topic.enable is not set to true.

➜  kafka_2.11-0.8.3-SNAPSHOT git:(trunk) ✗ ./bin/kafka-topics.sh --zookeeper localhost:2181 --list
➜  kafka_2.11-0.8.3-SNAPSHOT git:(trunk) ✗

The point is to have delete.topic.enable=true in config/server.properties that you use to start a Kafka broker.

➜  kafka_2.11-0.8.3-SNAPSHOT git:(trunk) ✗ grep delete.topic.enable config/server.properties
delete.topic.enable=true

You can also ensure the setting be true in a broker's log:

➜  kafka_2.11-0.8.3-SNAPSHOT git:(trunk) ✗ ./bin/kafka-server-start.sh config/server.properties
[2015-07-24 22:33:26,184] INFO KafkaConfig values:
        ...
        delete.topic.enable = true

Answer 13

The correct answer is actually the following. HOT off the Kafka-user's group email distribution list:

François Langelier *@gmail.com über kafka.apache.org 05:57 (vor 1 Stunde)

an users The delete topic isn't working ATM

I think it will be available in the next release https://issues.apache.org/jira/browse/KAFKA-1397