CODEWITHSUNDEEP
cassandradatastax-enterprise
noorul
noorul

Reputation: 1353

What is the ideal way to merge two Cassandra clusters with same keyspace into one?

We have two clusters X and Y with same keyspaces but distinct data sets. We are planning to merge these into single cluster. What would be ideal steps to achieve this without downtime for applications? We have time series data stream continuously writing to Cassandra.

We have ruled out export/import as that will make us lose data during the time of copy.

We also ruled out sstableloader as that is not reliable. It fails often and there is not way to start from where it failed. Also it has same issue mentioned above.

Upvotes: 0

Views: 1031

Answers (1)

phact
phact

Reputation: 7305

Do double writes (to both clusters) then any of the methods above will work.

1) A spark job is probably best if you have a ton of data.

2) import export is not as good as Brian's cassandra-loader so maybe give that a try https://github.com/brianmhess/cassandra-loader

We also ruled out sstableloader as that is not reliable. It fails often and there is not way to start from where it failed. Also it has same issue mentioned above.

I'm assuming your writes are idempotent so it's not a huge deal if you need to run the job over.

Upvotes: 1

Related Questions