CODEWITHSUNDEEP
cassandra
Leonid
Leonid

Reputation: 195

Adding new DC to a cluster that uses initial_token/allocate_tokens_for_keyspace allocation scheme

We have a 3 DC cluster running Cassandra 3.10. Each DC has 24 nodes total with 8 tokens per node and 3 seed nodes. We use Murmur3Partitioner.

In order to ensure better data distribution, the cluster was created using tokens allocation approach where you manually specify initial_token for seed nodes and use allocate_tokens_for_keyspace for non seed nodes.

Now I need to add another DC to the cluster using the same tokens allocation approach, but I can't figure out how to calculate initial_token for the new seed nodes. My naive approach was to copy token values from one of the existing DCs only to discover that initial token values must be unique across the whole cluster.
So, now I'm kinda lost on how to proceed. Any help will be appreciated, thanks.

Upvotes: 1

Views: 171

Answers (1)

Aaron
Aaron

Reputation: 57798

Correct, token range assignments must be unique.

In the days before VNodes and automatic token range calculations, we used to have to calculate token ranges manually. For multiple data centers, we would use the same token ranges offset by 1.

Example: If you use num_tokens: 4 and have 6 nodes in your DC, your nodes' token ranges might look something like this:

node1:  8454757700450209793, -5380300354831952895, -768614336404564991, 3843071682022821889 
node2: -9223372036854775807, -4611686018427387903, 1,                   4611686018427387905
node3: -8454757700450211199, -3843071682022823935, 768614336404563969,  5380300354831951873
node4: -7686143364045646591, -3074457345618258943, 1537228672809127937, 6148914691236515841
node5: -6917529027641081855, -2305843009213693951, 2305843009213693953, 6917529027641081857
node6: -6148914691236517375, -1537228672809129983, 3074457345618257921, 7686143364045645825

If you added a second DC, then the ranges for the new 6 nodes in the other DC would look like this:

node1:  8454757700450209794, -5380300354831952894, -768614336404564990, 3843071682022821890 
node2: -9223372036854775806, -4611686018427387902, 2,                   4611686018427387906
node3: -8454757700450211198, -3843071682022823934, 768614336404563970,  5380300354831951874
node4: -7686143364045646590, -3074457345618258942, 1537228672809127938, 6148914691236515842
node5: -6917529027641081854, -2305843009213693950, 2305843009213693954, 6917529027641081858
node6: -6148914691236517374, -1537228672809129982, 3074457345618257922, 7686143364045645826

If you'd have to do this for another DC, then you would offset their starting token ranges by 2.

Upvotes: 2

Related Questions