CODEWITHSUNDEEP
cassandra
Jean.ChangYi
Jean.ChangYi

Reputation: 1

Cassandra 2.1.17: sstableloader miss tombstone with -cph3

While using Cassandra 2.1.17, I encounter a problem in this scenario:

  1. backup and upload data to aws s3 from the production environment (3-nodes cluster)
  2. download these data local
  3. restore these data to the local cassandra (single-node) with this command: /$cassandra_path/bin/sstableloader -d $local_ip -cph 3 $sstable_path
  4. do some calculation with the local data

Sometimes I can get lot of extra data in local cassandra cluster which can not be found in the online cluster.

I pick one of these records, using the sstable2json to explain the sstables, I can find a tombstone record with the exactly key in the file where I download from s3 directly, whose timestamp is greater than the record's. That means this record has been deleted and for some reason, it revived during restore to the local cluster.

What's more, I search the local cluster's data, only the data record is still here while I can't find the tombstone record.

Have anyone met this before? Is't a sstableloader's bug? If so, how can I prevent it?

Upvotes: 0

Views: 66

Answers (1)

Steffen Winther Sørensen
Steffen Winther Sørensen

Reputation: 361

Just wondering about how well time might be synchronized between prod cluster and local node and if this could influence on TS revival...

Upvotes: 0

Related Questions