CODEWITHSUNDEEP
nginxsslkubernetescak3s
Jonas Grønbek
Jonas Grønbek

Reputation: 2019

kubernetes k3s agent can't connect to master CA/SSL error

I am trying to deploy an application with k3s kubernetes. Currently I have two master nodes behind a load-balancer, and I have some issues connecting worker nodes to them. All nodes and the load-balancer runs in seperate vms.

The load balancer is a nginx server with the following configuration.

load_module /usr/lib/nginx/modules/ngx_stream_module.so;
events {}
stream {
  upstream k3s_servers {
    server {master_node1_ip}:6443;
    server {master_node2_ip}:6443;
  }
  server {
    listen 6443;
    proxy_pass k3s_servers;
  }
}

the master nodes connects through the load-balancer, and seemingly it works as expected.

ubuntu@ip-172-31-20-78:/$ sudo k3s kubectl get nodes
NAME               STATUS   ROLES                  AGE   VERSION
ip-172-31-33-183   Ready    control-plane,master   81m   v1.20.2+k3s1
ip-172-31-20-78    Ready    control-plane,master   81m   v1.20.2+k3s1

However the worker nodes yields an error about the SSL certificate?

sudo systemctl status k3s-agent
● k3s-agent.service - Lightweight Kubernetes
     Loaded: loaded (/etc/systemd/system/k3s-agent.service; enabled; vendor preset: enabled)
     Active: active (running) since Sun 2021-01-24 15:54:10 UTC; 19min ago
       Docs: https://k3s.io
    Process: 3065 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, status=0/SUCCESS)
    Process: 3066 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
   Main PID: 3067 (k3s-agent)
      Tasks: 6
     Memory: 167.3M
     CGroup: /system.slice/k3s-agent.service
             └─3067 /usr/local/bin/k3s agent
Jan 24 16:12:23 ip-172-31-27-179 k3s[3311]: time="2021-01-24T16:34:02.483557102Z" level=info msg="Running load balancer 127.0.0.1:39357 -> [104.248.34.
Jan 24 16:12:23 ip-172-31-27-179 k3s[3067]: time="2021-01-24T16:12:23.313819380Z" level=error msg="failed to get CA certs: Get \"https://127.0.0.1:339

level=error msg="failed to get CA certs: Get "https://127.0.0.1:39357/cacerts": EOF"

if I try to change K3S_URL in /etc/systemd/system/k3s-agent.service.env to use http, I get an error saying that only https is accepted.

Upvotes: 13

Views: 10953

Answers (3)

Lherben G
Lherben G

Reputation: 383

Please review the resources allocated to your virtual machine (VM) and ensure that they meet the minimum requirements.

In the screenshot provided, you can see that I encountered the same error. Adjusting my memory fixed the issue.

enter image description here

Upvotes: 0

J Pod
J Pod

Reputation: 31

Try to curl the certs on the master node:

curl -vk https://<master_ip>:6443/cacerts

If you are unable to curl the certs, it is possible your master node is blocking requests on port 6443. From the master node, you can use IP tables to accept inputs on port 6443.

iptables -A INPUT -p tcp --dport 6443 -j ACCEPT

Try to curl again from the other nodes after setting this rule.

Upvotes: 2

clbx
clbx

Reputation: 184

Using the IP Address instead of the hostname in k3s-agent.service.env works for me. Not really a solution as much as a workaround.

/etc/systemd/system/k3s-agent.service.env

K3S_TOKEN='<token>'
K3S_URL='192.168.xxx.xxx:6443'

Upvotes: 3

Related Questions