Reputation: 115
Basically, I am running microk8s
with metallb
on a single node (Ubuntu Server 23.04 on a spare computer) and trying to get a WireGuard + qBittorrent + Firefox stack to run on it. I am encountering some issues, though: the pod spins up without any issues and it seems to have connectivity, but I cannot access the various UIs from my browser.
$ sudo microk8s kubectl get pods -n wg-vpn
NAME READY STATUS RESTARTS AGE
wireguard-5cdb4f956d-9h6vh 3/3 Running 0 3m25s
$ sudo microk8s kubectl describe pods wireguard-5cdb4f956d-9h6vh -n wg-vpn
Name: wireguard-5cdb4f956d-9h6vh
Namespace: wg-vpn
Priority: 0
Service Account: default
Node: homeserver/10.0.0.100
Start Time: Sun, 03 Sep 2023 14:07:27 +0200
Labels: name=wireguard
pod-template-hash=5cdb4f956d
Annotations: cni.projectcalico.org/containerID: a80cff8081b0c85eceb40870448ddd19c21f235773ccfc662a7be9dc9f237c72
cni.projectcalico.org/podIP: 10.1.242.125/32
cni.projectcalico.org/podIPs: 10.1.242.125/32
Status: Running
IP: 10.1.242.125
IPs:
IP: 10.1.242.125
Controlled By: ReplicaSet/wireguard-5cdb4f956d
Init Containers:
wireguard-template-replacement:
Container ID: containerd://e9b5a16f95783efca8dbbffe6a7dd4df85d6cfdc48b65e6e15d17f4b8fee54d2
Image: busybox
Image ID: docker.io/library/busybox@sha256:3fbc632167424a6d997e74f52b878d7cc478225cffac6bc977eedfe51c7f4e79
Port: <none>
Host Port: <none>
Command:
sh
-c
ENI=$(ip route get 8.8.8.8 | grep 8.8.8.8 | awk '{print $5}'); sed "s/ENI/$ENI/g" /etc/wireguard-secret/wg0.conf.template > /etc/wireguard/wg0.conf; chmod 400 /etc/wireguard/wg0.conf
State: Terminated
Reason: Completed
Exit Code: 0
Started: Sun, 03 Sep 2023 14:07:29 +0200
Finished: Sun, 03 Sep 2023 14:07:29 +0200
Ready: True
Restart Count: 0
Environment: <none>
Mounts:
/etc/wireguard-secret/ from wireguard-secret (rw)
/etc/wireguard/ from wireguard-config (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-wn8gr (ro)
Containers:
wireguard:
Container ID: containerd://f78b88c51713cfbd3bfbd7625f2572a8c39ea5e42c591e03d93edb3cf6abeadc
Image: linuxserver/wireguard:latest
Image ID: docker.io/linuxserver/wireguard@sha256:e4a9eea65549f4da582801c9b0ac42ea15226ed8fc66a87ab3200eac4a26c32d
Port: 58471/TCP
Host Port: 0/TCP
State: Running
Started: Sun, 03 Sep 2023 14:07:31 +0200
Ready: True
Restart Count: 0
Limits:
cpu: 1
memory: 512Mi
Requests:
cpu: 250m
memory: 128Mi
Environment:
TZ: Europe/Oslo
PEERS: wg-vpn
Mounts:
/etc/wireguard/ from wireguard-config (ro)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-wn8gr (ro)
qbittorrent:
Container ID: containerd://c931a77b27ab0e7186ed248ed1a3d2843b6f12f8a1dfdf41f066f86d1b75f3d0
Image: linuxserver/qbittorrent:latest
Image ID: docker.io/linuxserver/qbittorrent@sha256:a8f09fbcaa67736e44d203ecde5e91fd73cf7c11f61f15aebf77063c8332782d
Port: <none>
Host Port: <none>
State: Running
Started: Sun, 03 Sep 2023 14:07:38 +0200
Ready: True
Restart Count: 0
Limits:
cpu: 2
memory: 2Gi
Requests:
cpu: 250m
memory: 128Mi
Environment Variables from:
qbittorrent-config ConfigMap Optional: false
Environment: <none>
Mounts:
/config from qbittorrent-config (rw)
/downloads from qbittorrent-downloads (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-wn8gr (ro)
firefox:
Container ID: containerd://3fc0d3ac07e568e56d6f31159dbe0c5e98b83d2e5cd141373d8c28d3fef4bdbf
Image: linuxserver/firefox:latest
Image ID: docker.io/linuxserver/firefox@sha256:e8831383070b6ccf971c6df580b1cc0c04bece32578b3b94348ddf507305c938
Port: <none>
Host Port: <none>
SeccompProfile: Unconfined
State: Running
Started: Sun, 03 Sep 2023 14:08:13 +0200
Ready: True
Restart Count: 0
Limits:
cpu: 2
memory: 2Gi
Requests:
cpu: 250m
memory: 128Mi
Environment Variables from:
firefox-config ConfigMap Optional: false
Environment: <none>
Mounts:
/config from firefox-config (rw)
/dev/shm from firefox-shm (rw)
/var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-wn8gr (ro)
Conditions:
Type Status
Initialized True
Ready True
ContainersReady True
PodScheduled True
Volumes:
wireguard-config:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium:
SizeLimit: <unset>
wireguard-secret:
Type: Secret (a volume populated by a Secret)
SecretName: wireguard
Optional: false
qbittorrent-config:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: qbittorrent-pvc-config
ReadOnly: false
qbittorrent-downloads:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: qbittorrent-pvc-downloads
ReadOnly: false
firefox-config:
Type: PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
ClaimName: firefox-pvc
ReadOnly: false
firefox-shm:
Type: EmptyDir (a temporary directory that shares a pod's lifetime)
Medium: Memory
SizeLimit: 1Gi
kube-api-access-wn8gr:
Type: Projected (a volume that contains injected data from multiple sources)
TokenExpirationSeconds: 3607
ConfigMapName: kube-root-ca.crt
ConfigMapOptional: <nil>
DownwardAPI: true
QoS Class: Burstable
Node-Selectors: <none>
Tolerations: node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 3m43s default-scheduler Successfully assigned wg-vpn/wireguard-5cdb4f956d-9h6vh to homeserver
Normal Pulling 3m42s kubelet Pulling image "busybox"
Normal Pulled 3m41s kubelet Successfully pulled image "busybox" in 848.400291ms (848.414206ms including waiting)
Normal Created 3m41s kubelet Created container wireguard-template-replacement
Normal Started 3m41s kubelet Started container wireguard-template-replacement
Normal Pulling 3m40s kubelet Pulling image "linuxserver/wireguard:latest"
Normal Pulled 3m39s kubelet Successfully pulled image "linuxserver/wireguard:latest" in 807.976341ms (807.991166ms including waiting)
Normal Created 3m39s kubelet Created container wireguard
Normal Started 3m39s kubelet Started container wireguard
Normal Pulling 3m39s kubelet Pulling image "linuxserver/qbittorrent:latest"
Normal Pulled 3m32s kubelet Successfully pulled image "linuxserver/qbittorrent:latest" in 6.937124943s (6.937133519s including waiting)
Normal Created 3m32s kubelet Created container qbittorrent
Normal Started 3m32s kubelet Started container qbittorrent
Normal Pulling 3m32s kubelet Pulling image "linuxserver/firefox:latest"
Normal Pulled 2m57s kubelet Successfully pulled image "linuxserver/firefox:latest" in 34.774499456s (34.774510875s including waiting)
Normal Created 2m57s kubelet Created container firefox
Normal Started 2m57s kubelet Started container firefox
$ sudo microk8s kubectl get svc -n wg-vpn
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
wireguard LoadBalancer 10.152.183.92 10.0.0.13 58471:30042/UDP 4m35s
qbittorrent-service LoadBalancer 10.152.183.36 10.0.0.14 8080:30653/TCP,6881:31222/TCP,6881:31222/UDP 4m35s
firefox-service LoadBalancer 10.152.183.179 10.0.0.15 3000:32611/TCP,3001:31031/TCP 4m35s
$ sudo microk8s kubectl exec -it wireguard-5cdb4f956d-9h6vh -n wg-vpn -- /bin/sh
Defaulted container "wireguard" out of: wireguard, qbittorrent, firefox, wireguard-template-replacement (init)
root@wireguard-5cdb4f956d-9h6vh:/# ping google.com
PING google.com (172.217.16.206) 56(84) bytes of data.
64 bytes from fra16s08-in-f206.1e100.net (172.217.16.206): icmp_seq=1 ttl=59 time=45.9 ms
64 bytes from fra16s08-in-f14.1e100.net (172.217.16.206): icmp_seq=2 ttl=59 time=45.6 ms
64 bytes from fra16s08-in-f14.1e100.net (172.217.16.206): icmp_seq=3 ttl=59 time=45.7 ms
--- google.com ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2003ms
rtt min/avg/max/mdev = 45.584/45.711/45.850/0.108 ms
Also, running curl ipconfig.io
returns the correct IP (the VPN IP).
I'm guessing everything is working as it should with regards to internet connectivity but I cannot seem to connect to the pod from my browser, which probably means that there is an issue with the connectivity to the pod from within my local network.
---
apiVersion: v1
kind: Namespace
metadata:
name: wg-vpn
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: qbittorrent-pv-config
namespace: wg-vpn
spec:
capacity:
storage: 1Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
hostPath:
path: "/tank/testing/qbittorrent"
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: qbittorrent-pv-downloads
namespace: wg-vpn
spec:
capacity:
storage: 4Ti
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
hostPath:
path: "/tank/testing/torrents"
---
apiVersion: v1
kind: PersistentVolume
metadata:
name: firefox-pv
namespace: wg-vpn
spec:
capacity:
storage: 1Gi
accessModes:
- ReadWriteOnce
persistentVolumeReclaimPolicy: Retain
hostPath:
path: "/tank/testing/firefox"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: qbittorrent-pvc-config
namespace: wg-vpn
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
volumeName: qbittorrent-pv-config
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: qbittorrent-pvc-downloads
namespace: wg-vpn
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 4Ti
volumeName: qbittorrent-pv-downloads
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: firefox-pvc
namespace: wg-vpn
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
volumeName: firefox-pv
---
apiVersion: v1
kind: Secret
metadata:
name: wireguard
namespace: wg-vpn
type: Opaque
stringData:
wg0.conf.template: |
[Interface]
Address = 10.13.128.89/24
ListenPort = 58471
PrivateKey = [REDACTED]
MTU = 1390
DNS = 1.1.1.1
PostUp = iptables -A FORWARD -i %i -j ACCEPT; iptables -A FORWARD -o %i -j ACCEPT; iptables -t nat -A POSTROUTING -o ENI -j MASQUERADE
PostUp = sysctl -w -q net.ipv4.ip_forward=1
PostDown = iptables -D FORWARD -i %i -j ACCEPT; iptables -D FORWARD -o %i -j ACCEPT; iptables -t nat -D POSTROUTING -o ENI -j MASQUERADE
PostDown = sysctl -w -q net.ipv4.ip_forward=0
[Peer]
PublicKey = [REDACTED]
AllowedIPs = 0.0.0.0/0
Endpoint = 37.120.153.72:1443
PersistentKeepalive = 25
---
apiVersion: v1
kind: ConfigMap
metadata:
name: qbittorrent-config
namespace: wg-vpn
data:
PUID: "1000"
PGID: "1000"
TZ: "Europe/Oslo"
WEBUI_PORT: "8080"
---
apiVersion: v1
kind: ConfigMap
metadata:
name: firefox-config
namespace: wg-vpn
data:
PUID: "1000"
PGID: "1000"
TZ: "Europe/Oslo"
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: wireguard
namespace: wg-vpn
spec:
selector:
matchLabels:
name: wireguard
template:
metadata:
labels:
name: wireguard
spec:
initContainers:
# The exact name of the network interface needs to be stored in the
# wg0.conf WireGuard configuration file, so that the routes can be
# created correctly.
# The template file only contains the "ENI" placeholder, so when
# bootstrapping the application we'll need to replace the placeholder
# and create the actual wg0.conf configuration file.
- name: "wireguard-template-replacement"
image: "busybox"
command:
[
"sh",
"-c",
'ENI=$(ip route get 8.8.8.8 | grep 8.8.8.8 | awk ''{print $5}''); sed "s/ENI/$ENI/g" /etc/wireguard-secret/wg0.conf.template > /etc/wireguard/wg0.conf; chmod 400 /etc/wireguard/wg0.conf',
]
volumeMounts:
- name: wireguard-config
mountPath: /etc/wireguard/
- name: wireguard-secret
mountPath: /etc/wireguard-secret/
containers:
- name: "wireguard"
image: "linuxserver/wireguard:latest"
resources:
requests:
memory: "128Mi"
cpu: "250m"
limits:
memory: "512Mi"
cpu: "1000m"
ports:
- containerPort: 58471
env:
- name: "TZ"
value: "Europe/Oslo"
# Keep the PEERS environment variable to force server mode
- name: "PEERS"
value: "wg-vpn"
volumeMounts:
- name: wireguard-config
mountPath: /etc/wireguard/
readOnly: true
securityContext:
privileged: true
capabilities:
add:
- NET_ADMIN
- name: qbittorrent
image: linuxserver/qbittorrent:latest
resources:
requests:
memory: "128Mi"
cpu: "250m"
limits:
memory: "2048Mi"
cpu: "2000m"
envFrom:
- configMapRef:
name: qbittorrent-config
volumeMounts:
- name: qbittorrent-config
mountPath: /config
- name: qbittorrent-downloads
mountPath: /downloads
- name: firefox
image: linuxserver/firefox:latest
resources:
requests:
memory: "128Mi"
cpu: "250m"
limits:
memory: "2048Mi"
cpu: "2000m"
securityContext:
seccompProfile:
type: Unconfined
envFrom:
- configMapRef:
name: firefox-config
volumeMounts:
- name: firefox-config
mountPath: /config
- name: firefox-shm
mountPath: /dev/shm
volumes:
- name: wireguard-config
emptyDir: {}
- name: wireguard-secret
secret:
secretName: wireguard
- name: qbittorrent-config
persistentVolumeClaim:
claimName: qbittorrent-pvc-config
- name: qbittorrent-downloads
persistentVolumeClaim:
claimName: qbittorrent-pvc-downloads
- name: firefox-config
persistentVolumeClaim:
claimName: firefox-pvc
- name: firefox-shm
emptyDir:
medium: Memory
sizeLimit: 1Gi
imagePullSecrets:
- name: docker-registry
---
apiVersion: v1
kind: Service
metadata:
name: wireguard
namespace: wg-vpn
spec:
type: LoadBalancer
ports:
- name: wireguard
port: 58471
protocol: UDP
targetPort: 58471
selector:
name: wireguard
---
apiVersion: v1
kind: Service
metadata:
name: wireguard
namespace: wg-vpn
spec:
type: LoadBalancer
ports:
- name: wireguard
port: 58471
protocol: UDP
targetPort: 58471
selector:
name: wireguard
---
apiVersion: v1
kind: Service
metadata:
name: qbittorrent-service
namespace: wg-vpn
spec:
selector:
app: qbittorrent
ports:
- name: qbittorrent-webui
port: 8080
targetPort: 8080
- name: qbittorrent-tcp
protocol: TCP
port: 6881
targetPort: 6881
- name: qbittorrent-udp
protocol: UDP
port: 6881
targetPort: 6881
type: LoadBalancer
---
apiVersion: v1
kind: Service
metadata:
name: firefox-service
namespace: wg-vpn
spec:
selector:
app: firefox
ports:
- name: firefox-port1
port: 3000
targetPort: 3000
- name: firefox-port2
port: 3001
targetPort: 3001
type: LoadBalancer
Appreciate any input on this.
Upvotes: 0
Views: 143