CODEWITHSUNDEEP
kubernetescalico
Karthik Prasad
Karthik Prasad

Reputation: 10034

Kubernetes cluster with firewall enabled on CentOS(calico) not working

I've bringing up Kubernetes cluster with calico as CNI on CentOS 7 with firewall enabled. I've master and worker nodes. I was able to bring up cluster and able to list the nodes and Kubernetes system pods, all are working fine. However I'm unable to perform dns lookup.

System configuration

Kubernetes: 1.21.1
Calico: 3.19.1
Docker: 20.10.5
CentOS 7.9
IPVS enabled
Using VXLAN based network in Calico

Firewall configuration

The Problem is DNS lookup doesn't work. Tried following steps to identify iptables lookup.

    apiVersion: v1
    kind: Pod
    metadata:
      name: dnsutils
      namespace: default
    spec:
      containers:
      - name: dnsutils
        image: gcr.io/kubernetes-e2e-test-images/dnsutils:1.3
        command:
          - sleep
          - "3600"
        imagePullPolicy: IfNotPresent
      restartPolicy: Always

    firewall-cmd --set-log-denied=all
    firewall-cmd --reload

    $kubectl exec -it dnsutils -- nslookup kubernetes.default

result:

   ;; connection timed out; no servers could be reached

   command terminated with exit code 1

    dmesg | grep -i reject

result:

    [ 5556.708338] FINAL_REJECT: IN=calib3c61c3cba9 OUT= MAC=ee:ee:ee:ee:ee:ee:de:c8:d5:97:58:87:08:00 SRC=10.244.212.65 DST=10.96.0.10 LEN=90 TOS=0x00 PREC=0x00 TTL=64 ID=49835 PROTO=UDP SPT=52743 DPT=53 LEN=70
    [ 5561.707815] FINAL_REJECT: IN=calib3c61c3cba9 OUT= MAC=ee:ee:ee:ee:ee:ee:de:c8:d5:97:58:87:08:00 SRC=10.244.212.65 DST=10.96.0.10 LEN=90 TOS=0x00 PREC=0x00 TTL=64 ID=52640 PROTO=UDP SPT=52743 DPT=53 LEN=70
    [ 5566.708055] FINAL_REJECT: IN=calib3c61c3cba9 OUT= MAC=ee:ee:ee:ee:ee:ee:de:c8:d5:97:58:87:08:00 SRC=10.244.212.65 DST=10.96.0.10 LEN=90 TOS=0x00 PREC=0x00 TTL=64 ID=54942 PROTO=UDP SPT=52743 DPT=53 LEN=70

Upvotes: 0

Views: 6699

Answers (3)

Kiu Huang
Kiu Huang

Reputation: 21

cali* interfaces can be achieved using "+" wildcard:

name=kubeAccept
sudo firewall-cmd --permanent --new-zone=${name}
sudo firewall-cmd --permanent --zone=${name} --set-target=ACCEPT
sudo firewall-cmd --permanent --zone=${name} --add-interface=vxlan.calico
sudo firewall-cmd --permanent --zone=${name} --add-interface="cali+"

Tested and works in AlmaLinux 9 or Ubuntu 22.04 LTS

Upvotes: 2

Nate Goodman
Nate Goodman

Reputation: 1

I was able to get this working with the following steps.

sudo firewall-cmd --permanent --new-zone={name}
sudo firewall-cmd --permanent --zone={name} --set-target=ACCEPT
sudo firewall-cmd --permanent --zone={name} --add-interface=vxlan.calico

Then I looped through the calico network interfaces

for i in $(ip a | grep cali | awk -F":" '{print $2}' | awk -F"@" '{print $1}') 
do 
  sudo firewall-cmd --permanent --zone={name} --add-interface="$i"
done
sudo firewall-cmd --reload

Upvotes: 0

Digvijay
Digvijay

Reputation: 1

Faced a similar issue recently and it took us time to find what the actual problem is.

Problem

Even when you add ports for calico it does not work because, when you explicitly add port to firewall it blocks all other ports across interfaces. And calico creates a new network interface(vxlan.calico) which needs to be added to firewalld

Symptom

One symptom we noticed due to this was, we were able to ping pods across nodes. So icmp was working but tcp/udp did not work. Most likely you will get the same symptom.

Solution

You will need to add the network interface created by calico (check ifconfig to get exact name) to a trusted zone in firewall or create a new zone for it.

Create new zone and adding interface to it:

sudo firewall-cmd --permanent --new-zone={name}
sudo firewall-cmd --permanent --zone={name} --set-target=ACCEPT
sudo firewall-cmd --permanent --zone={name} --add-interface=vxlan.calico
sudo firewall-cmd --reload

Note - you will need to do above steps for master as well as all nodes.

Upvotes: 0

Related Questions