CODEWITHSUNDEEP
kubernetes
sergey_o
sergey_o

Reputation: 309

Can Kubernetes pod associate multiple IP addresses?

SHORT QUESTION:

Is there a way for a Kubernetes pod to associate multiple IP addresses?

Even if only loopback ones?

LONGER EXPLANATION:

I have an application that needs to be deployed in a Kubernetes pod and makes use of Cassandra. Cassandra itself is located on premise behind the firewall that, for administrative reasons, cannot be opened for direct IP access from an external cloud where the K8S landscape is hosted. Instead, I have to develop a relay that goes through a custom tunnel.

Cassandra driver inside the application will be pointed not to the real Cassandra cluster, but to the relay, which then will tunnel a connection.

I'd very much prefer to run the relay inside the pod itself (even better right inside the app container), to minimize the number of data traversals, since data transmission rate will be quite high, and also to minimize the number of distinct failure points and components to manage, and also to provide a measure of co-scaling with app replicas (the app is autoscaled, potentially to a large number of replicas).

The problem however is that Cassandra driver makes connection to every node in Cassandra cluster by node IP address, e.g. if Cassandra cluster is three nodes, then the driver connects to node1:9042, node2:9042 and node3:9042. Port number is force-shared by all the connections. The driver does not allow to specify let us say node1:9042, node2:9043 and node3:9044. Thus, I cannot have the driver connect to thispod:9042, thispod:9043 and thispod:9044. If it were possible, I could have a relay to run inside the container, listen on three ports, and then forward the connections. However, because of Cassandra driver limitations, relay listening endpoints must have distinct IP addresses (and I'd rather avoid having to make a custom-modified version of the driver to lift this restriction).

Which brings us to a question: is it possible for a pod to associate additional IP addresses?

The type of address does not matter much, as long as it is possible within a container or pod to send data to this address and to receive from it. Communication is essentially loopback within the contaner, or pod. If it were a non-containerized environment but plain Linux VM, I could create additional loopback interfaces which would have solved the problem. But inside the container interfaces cannot be created.

Is there a way to make Kubernetes to associate additional IP addresses to the pod?

Upvotes: 3

Views: 6989

Answers (1)

kool
kool

Reputation: 3613

To associate additional IP addresses with pod you can use Multus CNI. It allows you to attach multiple network interfaces to pod. It required a default CNI for pod-to-pod communication (i.e Calico, Flannel).

How it works is it creates a NetworkAttachmentDefinition CRD, which then you add in annotation field in pod manifest. Besides that you can define default routes for the interfaces. Example usage:

CRD definition

apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
  name: macvlan-conf
spec:
  config: '{
      "cniVersion": "0.3.0",
      "type": "macvlan",
      "master": "eth0",
      "mode": "bridge",
      "ipam": {
        "type": "host-local",
        "subnet": "192.168.1.0/24",
        "rangeStart": "192.168.1.200",
        "rangeEnd": "192.168.1.216",
        "routes": [
          { "dst": "0.0.0.0/0" }
        ],
        "gateway": "192.168.1.1"
      }
    }'

Pod manifest:

apiVersion: v1
kind: Pod
metadata:
  name: samplepod
  annotations:
    k8s.v1.cni.cncf.io/networks: macvlan-conf
spec:
  containers:
  - name: samplepod
    command: ["/bin/ash", "-c", "trap : TERM INT; sleep infinity & wait"]
    image: alpine

And when you exec into the pod when it's running you can see that an additional interface was created:

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
2: eth0@if7: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1440 qdisc noqueue state UP 
    link/ether 86:75:ed:87:a1:1a brd ff:ff:ff:ff:ff:ff
    inet 192.168.171.65/32 scope global eth0
       valid_lft forever preferred_lft forever
3: net1@tunl0: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1460 qdisc noqueue state UP 
    link/ether 1a:58:6e:88:fb:f5 brd ff:ff:ff:ff:ff:ff
    inet 192.168.1.200/24 scope global net1
       valid_lft forever preferred_lft forever

Upvotes: 6

Related Questions