CODEWITHSUNDEEP
cassandracluster-computing
Rajiv Srivastava
Rajiv Srivastava

Reputation: 31

4 node Cluster Query

In a 4 node cluster 1.1.10, if we want to change underlying storage of one node, I am thinking of adopting following steps. Is this correct?

  1. Take snapshop of the disks
  2. Take the node down and take final snapshot and swap the storage
  3. Bring the node up
  4. Cassandra will automatically balance/catch up on the node

Step 2 is expected to take around 1 Hr. During this time cluster will take care of the customer queries.

Is there anything else that needs to be done?

Also for step 4, what happens if any users queries the data that resides on the node whose storage was swapped.

Upvotes: 1

Views: 124

Answers (1)

piotrwest
piotrwest

Reputation: 2166

Wouldn't it be better to add a new node, with appropriate storage, wait until Cassandra balance the cluster, then decommission the old node? Zero downtime, no need of snapshots taking. You can make a release from source - Cassandra 1.1.10.

Still, if you want to do it your way:

  • make sure you know your enemy - Cassandra will use hinted handoff mechanizm - check max_hint_window_in_ms value
  • to be sure to avoid data resurrection, run nodetool repair on all nodes, check GCGraceSeconds value
  • make sure your client's are using appropriate consistency level - ConsistencyLevel.ALL isn't good in this case

Upvotes: 1

Related Questions