CODEWITHSUNDEEP
cassandradatastaxdatastax-enterprisecassandra-2.1tombstone
Vikas Lalwani
Vikas Lalwani

Reputation: 43

Cassandra tombstones not deleted a month after actual record TTL

Running into an issue with DSE 4.7.

The tombstones are not being deleted even after compactions, cleanup, rebuild_index and repair. records have a 15 day ttl.

sstablemetadata output suggests that there are 90% tombstones

Any ideas?

sstablemetadata output

SSTable: ./abcd-abcd-ka-478675
Partitioner: org.apache.cassandra.dht.Murmur3Partitioner
Bloom Filter FP chance: 0.010000
Minimum timestamp: 1527521280829593
Maximum timestamp: 1527596173976435
SSTable max local deletion time: 1528892173
Compression ratio: 0.36967428395684393
Estimated droppable tombstones: 0.9073013816277629
SSTable Level: 0
Repaired at: 0
ReplayPosition(segmentId=1520529283052, position=4626679)
Estimated tombstone drop times:%n
1528817679:  18318196
1528818619:  20753822
1528819513:  24176310
.
.
.

Count               Row Size        Cell Count
1                          0                 0
2                          0           1752560
3                          0                 0
4                          0           6355421
5                          0                 0
6                          0            687302
7                          0                 0
8                          0            529613
10                         0            444801
12                         0            410107
14                         0            456011
17                         0           1347893
20                         0            184960
24                         0            152814
.
.
.
770                  1347893               137
924                   184960               109
1109                  220403                68
1331                  121620                86
1597                 2044030               102
1916                  185601               195
2299                  184816            158273
2759                  868754                 0
3311                   62795                 0
3973                    1668                 0
4768                    2143                 0
5722                 1812541                 0
6866                     828                 0
.
.
.             
Ancestors: [476190, 474027, 475201, 478160]
Estimated cardinality: 20059264

Upvotes: 2

Views: 1120

Answers (2)

Techturff
Techturff

Reputation: 1

Tombstones gets deleted after normal compaction. But, still sometime you find stale data (even in prod)in tombstone.The reason could be out of all the nodes in that cluster one is down and the data from tombstone did not got deleted because of that node. Also sometimes null values are inserted in primary key causing tombstone data.

Upvotes: 0

Payal
Payal

Reputation: 564

Cassandra marks TTL data with a tombstone after the requested amount of time has expired. A tombstone exists for gc_grace_seconds. After data is marked with a tombstone, the data is automatically removed during the normal compaction process. you can try to run major compaction to evict tombstone out.

Upvotes: 1

Related Questions