CODEWITHSUNDEEP
cassandracassandra-3.0
one.eye
one.eye

Reputation: 133

TTL expired sstables not getting dropped on Cassandra Nodes

I am inserting data in below table with TTL of 2 days. SSTABLES with timestamp older than 2 days should have been dropped but that is not happening. There are no deletes and no updates. Window size is 1 hour and gc_grace_seconds is 7200. read_repair has been set to 0.0

CREATE TABLE events."290" (
    key text PRIMARY KEY,
    raw_log text
) WITH COMPACT STORAGE
    AND bloom_filter_fp_chance = 0.01
    AND caching = {'keys': 'ALL', 'rows_per_partition': 'NONE'}
    AND comment = ''
    AND compaction = {'class': 'org.apache.cassandra.db.compaction.TimeWindowCompactionStrategy', 'compaction_window_size': '1', 'compaction_window_unit': 'HOURS', 'max_threshold': '64', 'min_threshold': '32'}
    AND compression = {'chunk_length_in_kb': '64', 'class': 'org.apache.cassandra.io.compress.SnappyCompressor'}
    AND crc_check_chance = 1.0
    AND dclocal_read_repair_chance = 0.0
    AND default_time_to_live = 0
    AND gc_grace_seconds = 7200
    AND max_index_interval = 2048
    AND memtable_flush_period_in_ms = 0
    AND min_index_interval = 128
    AND read_repair_chance = 0.0
    AND speculative_retry = 'NONE';

On restarting cassandra the expired sstables got removed as expected.

Should I be setting unchecked_tombstone_compaction = true? Ideally I would like the sstables to be dropped completely rather than go through multiple compactions to get removed.

Earlier I was using DateTieredCompaction and cassandra 2.2. Everything was working fine. Upgraded to cassandra 3.0.13 and observed above behaviour. Changed compaction from DTCS to TWCS and still the same behaviour.

If on restarting the expired sstables were not getting removed, I would have considered case of overlapping sstables https://issues.apache.org/jira/browse/CASSANDRA-13418

Upvotes: 1

Views: 1202

Answers (1)

Michiel Saelen
Michiel Saelen

Reputation: 1

We had similar issues where SSTables were not being dropped. This was because some data got updated with the same primary key. Therefore, the newer SSTables had an entry with an older timestamp as it uses the timestamp from the initial insert with that primary key.

We solved it by enabling the unsafe aggressive stable expiration in the cassandra-env.sh file:

JVM_OPTS="$JVM_OPTS -Dcassandra.allow_unsafe_aggressive_sstable_expiration=true"

AND adding that option to the compaction strategy of those tables:

ALTER TABLE KEYSPACE_NAME.TABLE_NAME
with compaction = {
    'class' : 'TimeWindowCompactionStrategy',
    'unsafe_aggressive_sstable_expiration' : true
};

More info on how to troubleshoot and to solve it can be found here: https://docs.dataminer.services/user-guide/Troubleshooting/Procedures/Cassandra_expired_blocking_SSTables.html

Upvotes: 0

Related Questions