How to expose minikube ingress to internet

Question

I have install a minikube on my linux-laptop (Computer A)

and deploy some service on it :

$ minikube service list

NAMESPACE	NAME	TARGET PORT	URL
cert-manager	cert-manager	No node port
cert-manager	cert-manager-webhook	No node port
default	aws	5000	http://192.168.49.2:30728
default	azure	5000	http://192.168.49.2:31794
default	gcloud	5000	http://192.168.49.2:32367
default	hft	5000	http://192.168.49.2:30970
default	hftf	5000	http://192.168.49.2:30612
default	kubernetes	No node port
default	open	5000	http://192.168.49.2:31547
ingress-nginx	ingress-nginx-controller	http/80	http://192.168.49.2:31053
		https/443	http://192.168.49.2:30302
ingress-nginx	ingress-nginx-controller-admission	No node port
kube-system	kube-dns	No node port
kubernetes-dashboard	dashboard-metrics-scraper	No node port
kubernetes-dashboard	kubernetes-dashboard	No node port

As you can see there is 6 service create with fastAPI and that are behind an ingress. I have set my ingress with the host galigator.fun. Cert-manager is ready to fire, but for now TLS is comment in the ingress. galigator.fun is DNS name I own.

$ minikube addons enable ingress
$ kubectl describe ingress
Name:             multiplex-applications
Namespace:        default
Address:          192.168.49.2
Default backend:  default-http-backend:80 ()
Rules:
  Host           Path  Backends
  ----           ----  --------
  galigator.fun  
                 /v1/aws/(.+)            aws:5000 (172.17.0.11:5000)
                 /v1/azure/(.+)          azure:5000 (172.17.0.14:5000)
                 /v1/gcloud/(.+)         gcloud:5000 (172.17.0.13:5000)
                 /v1/deepml/best/(.+)    hftf:5000 (172.17.0.10:5000)
                 /v1/deepml/edge/(.+)    hft:5000 (172.17.0.3:5000)
                 /v1/deepml/razor/(.+)   open:5000 (172.17.0.8:5000)
Annotations:     cert-manager.io/issue-temporary-certificate: true
                 cert-manager.io/issuer: letsencrypt-staging
                 kubernetes.io/ingress.class: nginx
                 nginx.ingress.kubernetes.io/rewrite-target: /$1
Events:
  Type    Reason  Age                From                      Message
  ----    ------  ----               ----                      -------
  Normal  Sync    52m (x3 over 53m)  nginx-ingress-controller  Scheduled for sync

The error from Default backend seem not related to my problem...

If add an entry to /etc/hosts with minikube ip and galigator.fun then I can run my services without problem. But there are not available from Computer B not Computer C.

How to access my services from Computer B and Computer C ?

I don't want to change any configuration in computer B & C.

As I understand, the minikube ip isn't know from my Internet Box; it is not a physical equipement. My minikube run from my kubuntu-21.04 and is start with minikube start --ports=80,443 or just minikube start but I can't observe a difference.

Another thing that don't look to work.

$ kubectl port-forward --address 0.0.0.0 deployment/ingress-nginx-controller 80:80 --namespace ingress-nginx
Unable to listen on port 80: Listeners failed to create with the following errors: [unable to create listener: Error listen tcp4 0.0.0.0:80: bind: permission denied]
error: unable to listen on any of the requested ports: [{80 80}]

I have properly test the connectivity between Internet and Computer A for port 80 & 443 using an Apache. Since I have remove that server.

Answer 1

First as the doc explain I try to play with minikube startup options.

minikube start --listen-address='0.0.0.0' --ports=80,443

But as we can see it doesn't change anything:

$ docker container ls |grep minikube
0aafb00fd97d   gcr.io/k8s-minikube/kicbase:v0.0.25   "/usr/local/bin/entr…"   3 weeks ago    Up 22 minutes   127.0.0.1:49172->22/tcp, 127.0.0.1:49171->2376/tcp, 127.0.0.1:49170->5000/tcp, 127.0.0.1:49169->8443/tcp, 127.0.0.1:49168->32443/tcp   minikube

So if we investigate deeper on error of command :

kubectl port-forward -n ingress-nginx  --address 0.0.0.0 service/ingress-nginx-controller 80:80 443:443

we can find that binding to port 80 and 443 but they are below 1000 and require extra privilege/root. And kubectl isn't a root process, for goods reasons.

So an ugly solution that work could be to do something like that :

kubectl port-forward -n ingress-nginx  --address 0.0.0.0 service/ingress-nginx-controller 50000:80 50001:443

And then change the NAT rules in the InternetBox. But I don't like it because minikube is an implementation detail that should not impact another hardware.

So a less horrible solution is to give kubectl the write to bind to low number ports...

$ man 7 capabilities | grep -A 1 CAP_NET_BIND_SERVICE
       CAP_NET_BIND_SERVICE
              Bind a socket to Internet domain privileged ports (port numbers less than 1024).

So command should be :

sudo setcap CAP_NET_BIND_SERVICE=+eip /usr/bin/kubectl

and then

kubectl port-forward -n ingress-nginx  --address 0.0.0.0 service/ingress-nginx-controller 80:80 443:443

it works to...

But there is one major concern here, we have to redo the last command on every startup, and the last two ones on every upgrade...

If someone have a good answer, that could also work for a background server on a LAN, I would love it.

Port 80 is require only during the execution of the challenge from cert-manager. So a solution that could get close it, could also be interesting.