How can we add a second Cassandra DC running in a separate Openshift/Kubernetes cluster?

Question

We deployed 2 cassandra datacenters in 2 different openshift clusters (one for each openshift cluster). Each cassandra datacenter has one seed pod (pod-0)

We used bitnami helm-chart ( https://github.com/bitnami/charts/tree/master/bitnami/cassandra)

Now we would like to connect both cassandra datacentres in order to synchronize the data. How can we do it?

I suppose that we need to expose Cassandra traffic using an openshift route. But which port and service to expose/use?

I can see that we have:

• a service called myrelease-cassandra, type ClusterIP, targeting 9042/TCP (cql)
• a service called myrelease-cassandra-headless, Headless(no ip), targeting 7000/TCP (intra), 7001/TCP (tls),7199/TCP (jmx),9042/TCP (cql)

I tried a couple of solution but so far I didn't succeed: for example I can see from the logs that the Cassandra rings are trying to connect over the port 9042, but the Openshift routes are accessible through the port 443:

• if I set the external seed (basically the other openshift cluser route) with the port number (e.g. my-os-route:443) I see errors saying: host could not be resolved.
• if I do not set the port number I see a timeout because the port is 9042 instead of 443

we configured cassandra to use ssl, and the routes in openshift are accessible thorugh an F5 loadbalancer which targets all the infra nodes of opsnhift

UPDATE 1

Basically we would like to replicate this architecture (but with 2 k8s clusters and without the operator) https://itnext.io/managing-a-multi-site-cassandra-cluster-on-multiple-kubernetes-with-casskop-multicasskop-cf407c297701

or https://docs.k8ssandra.io/components/k8ssandra-operator/

UPDATE 2

Basically if there would be a way to configure the intra node port as 7000 but saying to cassandra to use another port to connect to other cassandra hosts it would work. Something like Elasticsearch where you have http.port and http.publish_port Elasticsearch configuration

Answer 1

Can you use hostPort instead of nodePort. I've been able to get two independent OpenShift K8s kubes connected into a single cluster, using cass-operator, but it required a bit of hijinks. Can you see this doc https://docs.google.com/document/d/1YuS0FaCKIu_Sa9XMDRSI17MqMOHqzScR6SO7XVBImz4/edit

Answer 2

As you already know, these are the three basic attributes required for nodes to form a cluster:

• the same cluster_name
• inter-node network connectivity for gossip
• common seeds

For two DCs in different K8s clusters to be able to form a cluster, you'll need to setup the first DC with bi-directional TCP network connectivity enabled on gossip port 7000 between the two K8s clusters.

You will also need to make sure that the pods are configured to use a service for the seeds list in cassandra.yaml. This seeds service must be exposed so that it is accessible to the other K8s cluster.

Once the first DC is operational, you will need to replicate the configuration on to the second DC in the other K8s cluster such that:

• the second DC has the same cluster_name
• uses the same seeds service as DC1 in the seeds list of the pods
• have network connectivity to gossip on port 7000

Ideally, the second DC also has a seeds service configured so they can be used by both DCs. Cheers!