Upgrade cassandra 2.1.19 cluster to 3.11.1

Question

I want to upgrade cassandra 2.1.19 cluster to 3.11.1 without downtime. Will 3.11.1 nodes work together with 2.1.19 nodes at the same time?

Answer 1

Key point will be how you connect to your cluster. You will need to try out on test systems if everything works from your application side doing the switch.

I recommend a two stop process in this case, migrate from 2.1.19 to 3.0.x - one node at atime.

For every node do the following (i said you need to test before before going to production right?):

• nodetool drain - wait for finish
• stop cassandra
• backup your configs, the old one wont work out of the box
• remove the cassandra package / tarball
• read about the java and other cassandra 3.x requirements and ensure you met them
• add the repo and install the 3.0.x package or tar ball
• some packages start the node immediately - you may have to stop them again
• make up the new config files (diff or something will be your friend, read the docs about the new options), one time only you should be able to resuse if on all the other nodes
• start cassandra (did I say test this on a test system?) and wait until the node has joined the ring again nodetool status
• upgrade your sstables with nodetool upgradesstables - almost always needed, dont skip this even if "something" works right now
• this upgrade tends to be really slow - it's just a single thread running rewriting all your data, so i/o will be a factor here
• all up and running -> go ahead to the next node and repeat

After that - upgrade 3.0.x to 3.11.x in the same fashion, add the new repo, configure for 3.11.x as for 3.0.x above and so on. But this time you can skip upgrading sstables as the format stays the same (but it wont harm if you do so).

Did I mention to do this on testing system first? One thing that will happen and may break things - older native protocols will be gone as well as rpc/ thrift.

Hope I didn't miss something ;)