telometto
telometto

Reputation: 115

Why can't I connect to the pod/containers when I go to their IP in my browser?

Basically, I am running microk8s with metallb on a single node (Ubuntu Server 23.04 on a spare computer) and trying to get a WireGuard + qBittorrent + Firefox stack to run on it. I am encountering some issues, though: the pod spins up without any issues and it seems to have connectivity, but I cannot access the various UIs from my browser.

[RELEVANT INFO]

Pod status

$ sudo microk8s kubectl get pods -n wg-vpn
NAME                         READY   STATUS    RESTARTS   AGE
wireguard-5cdb4f956d-9h6vh   3/3     Running   0          3m25s

Pod description

$ sudo microk8s kubectl describe pods wireguard-5cdb4f956d-9h6vh -n wg-vpn
Name:             wireguard-5cdb4f956d-9h6vh
Namespace:        wg-vpn
Priority:         0
Service Account:  default
Node:             homeserver/10.0.0.100
Start Time:       Sun, 03 Sep 2023 14:07:27 +0200
Labels:           name=wireguard
                  pod-template-hash=5cdb4f956d
Annotations:      cni.projectcalico.org/containerID: a80cff8081b0c85eceb40870448ddd19c21f235773ccfc662a7be9dc9f237c72
                  cni.projectcalico.org/podIP: 10.1.242.125/32
                  cni.projectcalico.org/podIPs: 10.1.242.125/32
Status:           Running
IP:               10.1.242.125
IPs:
  IP:           10.1.242.125
Controlled By:  ReplicaSet/wireguard-5cdb4f956d
Init Containers:
  wireguard-template-replacement:
    Container ID:  containerd://e9b5a16f95783efca8dbbffe6a7dd4df85d6cfdc48b65e6e15d17f4b8fee54d2
    Image:         busybox
    Image ID:      docker.io/library/busybox@sha256:3fbc632167424a6d997e74f52b878d7cc478225cffac6bc977eedfe51c7f4e79
    Port:          <none>
    Host Port:     <none>
    Command:
      sh
      -c
      ENI=$(ip route get 8.8.8.8 | grep 8.8.8.8 | awk '{print $5}'); sed "s/ENI/$ENI/g" /etc/wireguard-secret/wg0.conf.template > /etc/wireguard/wg0.conf; chmod 400 /etc/wireguard/wg0.conf
    State:          Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Sun, 03 Sep 2023 14:07:29 +0200
      Finished:     Sun, 03 Sep 2023 14:07:29 +0200
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /etc/wireguard-secret/ from wireguard-secret (rw)
      /etc/wireguard/ from wireguard-config (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-wn8gr (ro)
Containers:
  wireguard:
    Container ID:   containerd://f78b88c51713cfbd3bfbd7625f2572a8c39ea5e42c591e03d93edb3cf6abeadc
    Image:          linuxserver/wireguard:latest
    Image ID:       docker.io/linuxserver/wireguard@sha256:e4a9eea65549f4da582801c9b0ac42ea15226ed8fc66a87ab3200eac4a26c32d
    Port:           58471/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Sun, 03 Sep 2023 14:07:31 +0200
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     1
      memory:  512Mi
    Requests:
      cpu:     250m
      memory:  128Mi
    Environment:
      TZ:     Europe/Oslo
      PEERS:  wg-vpn
    Mounts:
      /etc/wireguard/ from wireguard-config (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-wn8gr (ro)
  qbittorrent:
    Container ID:   containerd://c931a77b27ab0e7186ed248ed1a3d2843b6f12f8a1dfdf41f066f86d1b75f3d0
    Image:          linuxserver/qbittorrent:latest
    Image ID:       docker.io/linuxserver/qbittorrent@sha256:a8f09fbcaa67736e44d203ecde5e91fd73cf7c11f61f15aebf77063c8332782d
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Sun, 03 Sep 2023 14:07:38 +0200
    Ready:          True
    Restart Count:  0
    Limits:
      cpu:     2
      memory:  2Gi
    Requests:
      cpu:     250m
      memory:  128Mi
    Environment Variables from:
      qbittorrent-config  ConfigMap  Optional: false
    Environment:          <none>
    Mounts:
      /config from qbittorrent-config (rw)
      /downloads from qbittorrent-downloads (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-wn8gr (ro)
  firefox:
    Container ID:    containerd://3fc0d3ac07e568e56d6f31159dbe0c5e98b83d2e5cd141373d8c28d3fef4bdbf
    Image:           linuxserver/firefox:latest
    Image ID:        docker.io/linuxserver/firefox@sha256:e8831383070b6ccf971c6df580b1cc0c04bece32578b3b94348ddf507305c938
    Port:            <none>
    Host Port:       <none>
    SeccompProfile:  Unconfined
    State:           Running
      Started:       Sun, 03 Sep 2023 14:08:13 +0200
    Ready:           True
    Restart Count:   0
    Limits:
      cpu:     2
      memory:  2Gi
    Requests:
      cpu:     250m
      memory:  128Mi
    Environment Variables from:
      firefox-config  ConfigMap  Optional: false
    Environment:      <none>
    Mounts:
      /config from firefox-config (rw)
      /dev/shm from firefox-shm (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-wn8gr (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             True 
  ContainersReady   True 
  PodScheduled      True 
Volumes:
  wireguard-config:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     
    SizeLimit:  <unset>
  wireguard-secret:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  wireguard
    Optional:    false
  qbittorrent-config:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  qbittorrent-pvc-config
    ReadOnly:   false
  qbittorrent-downloads:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  qbittorrent-pvc-downloads
    ReadOnly:   false
  firefox-config:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  firefox-pvc
    ReadOnly:   false
  firefox-shm:
    Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
    Medium:     Memory
    SizeLimit:  1Gi
  kube-api-access-wn8gr:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age    From               Message
  ----    ------     ----   ----               -------
  Normal  Scheduled  3m43s  default-scheduler  Successfully assigned wg-vpn/wireguard-5cdb4f956d-9h6vh to homeserver
  Normal  Pulling    3m42s  kubelet            Pulling image "busybox"
  Normal  Pulled     3m41s  kubelet            Successfully pulled image "busybox" in 848.400291ms (848.414206ms including waiting)
  Normal  Created    3m41s  kubelet            Created container wireguard-template-replacement
  Normal  Started    3m41s  kubelet            Started container wireguard-template-replacement
  Normal  Pulling    3m40s  kubelet            Pulling image "linuxserver/wireguard:latest"
  Normal  Pulled     3m39s  kubelet            Successfully pulled image "linuxserver/wireguard:latest" in 807.976341ms (807.991166ms including waiting)
  Normal  Created    3m39s  kubelet            Created container wireguard
  Normal  Started    3m39s  kubelet            Started container wireguard
  Normal  Pulling    3m39s  kubelet            Pulling image "linuxserver/qbittorrent:latest"
  Normal  Pulled     3m32s  kubelet            Successfully pulled image "linuxserver/qbittorrent:latest" in 6.937124943s (6.937133519s including waiting)
  Normal  Created    3m32s  kubelet            Created container qbittorrent
  Normal  Started    3m32s  kubelet            Started container qbittorrent
  Normal  Pulling    3m32s  kubelet            Pulling image "linuxserver/firefox:latest"
  Normal  Pulled     2m57s  kubelet            Successfully pulled image "linuxserver/firefox:latest" in 34.774499456s (34.774510875s including waiting)
  Normal  Created    2m57s  kubelet            Created container firefox
  Normal  Started    2m57s  kubelet            Started container firefox

Pod services

$ sudo microk8s kubectl get svc -n wg-vpn
NAME                  TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)                                        AGE
wireguard             LoadBalancer   10.152.183.92    10.0.0.13     58471:30042/UDP                                4m35s
qbittorrent-service   LoadBalancer   10.152.183.36    10.0.0.14     8080:30653/TCP,6881:31222/TCP,6881:31222/UDP   4m35s
firefox-service       LoadBalancer   10.152.183.179   10.0.0.15     3000:32611/TCP,3001:31031/TCP                  4m35s

Pinging from inside the pod

$ sudo microk8s kubectl exec -it wireguard-5cdb4f956d-9h6vh -n wg-vpn -- /bin/sh
Defaulted container "wireguard" out of: wireguard, qbittorrent, firefox, wireguard-template-replacement (init)
root@wireguard-5cdb4f956d-9h6vh:/# ping google.com
PING google.com (172.217.16.206) 56(84) bytes of data.
64 bytes from fra16s08-in-f206.1e100.net (172.217.16.206): icmp_seq=1 ttl=59 time=45.9 ms
64 bytes from fra16s08-in-f14.1e100.net (172.217.16.206): icmp_seq=2 ttl=59 time=45.6 ms
64 bytes from fra16s08-in-f14.1e100.net (172.217.16.206): icmp_seq=3 ttl=59 time=45.7 ms
--- google.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 45.584/45.711/45.850/0.108 ms

Also, running curl ipconfig.io returns the correct IP (the VPN IP).

I'm guessing everything is working as it should with regards to internet connectivity but I cannot seem to connect to the pod from my browser, which probably means that there is an issue with the connectivity to the pod from within my local network.


[STACK]

Namespace

---
apiVersion: v1
kind: Namespace
metadata:
  name: wg-vpn

PV

---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: qbittorrent-pv-config
  namespace: wg-vpn
spec:
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  hostPath:
    path: "/tank/testing/qbittorrent"
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: qbittorrent-pv-downloads
  namespace: wg-vpn
spec:
  capacity:
    storage: 4Ti
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  hostPath:
    path: "/tank/testing/torrents"
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: firefox-pv
  namespace: wg-vpn
spec:
  capacity:
    storage: 1Gi
  accessModes:
    - ReadWriteOnce
  persistentVolumeReclaimPolicy: Retain
  hostPath:
    path: "/tank/testing/firefox"

PVC

---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: qbittorrent-pvc-config
  namespace: wg-vpn
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
  volumeName: qbittorrent-pv-config
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: qbittorrent-pvc-downloads
  namespace: wg-vpn
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 4Ti
  volumeName: qbittorrent-pv-downloads
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: firefox-pvc
  namespace: wg-vpn
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
  volumeName: firefox-pv

Secret

---
apiVersion: v1
kind: Secret
metadata:
  name: wireguard
  namespace: wg-vpn
type: Opaque
stringData:
  wg0.conf.template: |
    [Interface]
    Address = 10.13.128.89/24
    ListenPort = 58471
    PrivateKey = [REDACTED]
    MTU = 1390
    DNS = 1.1.1.1
    PostUp = iptables -A FORWARD -i %i -j ACCEPT; iptables -A FORWARD -o %i -j ACCEPT; iptables -t nat -A POSTROUTING -o ENI -j MASQUERADE
    PostUp = sysctl -w -q net.ipv4.ip_forward=1
    PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -D FORWARD -o %i -j ACCEPT; iptables -t nat -D POSTROUTING -o ENI -j MASQUERADE
    PostDown = sysctl -w -q net.ipv4.ip_forward=0

    [Peer]
    PublicKey = [REDACTED]
    AllowedIPs = 0.0.0.0/0
    Endpoint = 37.120.153.72:1443
    PersistentKeepalive = 25

ConfigMap

---
apiVersion: v1
kind: ConfigMap
metadata:
  name: qbittorrent-config
  namespace: wg-vpn
data:
  PUID: "1000"
  PGID: "1000"
  TZ: "Europe/Oslo"
  WEBUI_PORT: "8080"
---
apiVersion: v1
kind: ConfigMap
metadata:
  name: firefox-config
  namespace: wg-vpn
data:
  PUID: "1000"
  PGID: "1000"
  TZ: "Europe/Oslo"

Deployment

---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: wireguard
  namespace: wg-vpn
spec:
  selector:
    matchLabels:
      name: wireguard
  template:
    metadata:
      labels:
        name: wireguard
    spec:
      initContainers:
        # The exact name of the network interface needs to be stored in the
        # wg0.conf WireGuard configuration file, so that the routes can be
        # created correctly.
        # The template file only contains the "ENI" placeholder, so when
        # bootstrapping the application we'll need to replace the placeholder
        # and create the actual wg0.conf configuration file.
        - name: "wireguard-template-replacement"
          image: "busybox"
          command:
            [
              "sh",
              "-c",
              'ENI=$(ip route get 8.8.8.8 | grep 8.8.8.8 | awk ''{print $5}''); sed "s/ENI/$ENI/g" /etc/wireguard-secret/wg0.conf.template > /etc/wireguard/wg0.conf; chmod 400 /etc/wireguard/wg0.conf',
            ]
          volumeMounts:
            - name: wireguard-config
              mountPath: /etc/wireguard/
            - name: wireguard-secret
              mountPath: /etc/wireguard-secret/

      containers:
        - name: "wireguard"
          image: "linuxserver/wireguard:latest"
          resources:
            requests:
              memory: "128Mi"
              cpu: "250m"
            limits:
              memory: "512Mi"
              cpu: "1000m"
          ports:
            - containerPort: 58471
          env:
            - name: "TZ"
              value: "Europe/Oslo"
            # Keep the PEERS environment variable to force server mode
            - name: "PEERS"
              value: "wg-vpn"
          volumeMounts:
            - name: wireguard-config
              mountPath: /etc/wireguard/
              readOnly: true
          securityContext:
            privileged: true
            capabilities:
              add:
                - NET_ADMIN
        - name: qbittorrent
          image: linuxserver/qbittorrent:latest
          resources:
            requests:
              memory: "128Mi"
              cpu: "250m"
            limits:
              memory: "2048Mi"
              cpu: "2000m"
          envFrom:
            - configMapRef:
                name: qbittorrent-config
          volumeMounts:
            - name: qbittorrent-config
              mountPath: /config
            - name: qbittorrent-downloads
              mountPath: /downloads
        - name: firefox
          image: linuxserver/firefox:latest
          resources:
            requests:
              memory: "128Mi"
              cpu: "250m"
            limits:
              memory: "2048Mi"
              cpu: "2000m"
          securityContext:
            seccompProfile:
              type: Unconfined
          envFrom:
            - configMapRef:
                name: firefox-config
          volumeMounts:
            - name: firefox-config
              mountPath: /config
            - name: firefox-shm
              mountPath: /dev/shm
      volumes:
        - name: wireguard-config
          emptyDir: {}
        - name: wireguard-secret
          secret:
            secretName: wireguard
        - name: qbittorrent-config
          persistentVolumeClaim:
            claimName: qbittorrent-pvc-config
        - name: qbittorrent-downloads
          persistentVolumeClaim:
            claimName: qbittorrent-pvc-downloads
        - name: firefox-config
          persistentVolumeClaim:
            claimName: firefox-pvc
        - name: firefox-shm
          emptyDir:
            medium: Memory
            sizeLimit: 1Gi
      imagePullSecrets:
        - name: docker-registry

Service

---
apiVersion: v1
kind: Service
metadata:
  name: wireguard
  namespace: wg-vpn
spec:
  type: LoadBalancer
  ports:
    - name: wireguard
      port: 58471
      protocol: UDP
      targetPort: 58471
  selector:
    name: wireguard
---
apiVersion: v1
kind: Service
metadata:
  name: wireguard
  namespace: wg-vpn
spec:
  type: LoadBalancer
  ports:
    - name: wireguard
      port: 58471
      protocol: UDP
      targetPort: 58471
  selector:
    name: wireguard
---
apiVersion: v1
kind: Service
metadata:
  name: qbittorrent-service
  namespace: wg-vpn
spec:
  selector:
    app: qbittorrent
  ports:
    - name: qbittorrent-webui
      port: 8080
      targetPort: 8080
    - name: qbittorrent-tcp
      protocol: TCP
      port: 6881
      targetPort: 6881
    - name: qbittorrent-udp
      protocol: UDP
      port: 6881
      targetPort: 6881
  type: LoadBalancer
---
apiVersion: v1
kind: Service
metadata:
  name: firefox-service
  namespace: wg-vpn
spec:
  selector:
    app: firefox
  ports:
    - name: firefox-port1
      port: 3000
      targetPort: 3000
    - name: firefox-port2
      port: 3001
      targetPort: 3001
  type: LoadBalancer

Appreciate any input on this.

Upvotes: 0

Views: 143

Answers (0)

Related Questions