How to enable subdomain with GKE

Question

I have different Kubernetes deployment in GKE and I would like to access them from different external subdomains.

I tried to create 2 deployments with subdomain "sub1" and "sub2" and hostname "app" another deployment with hostname "app" and a service that expose it on the IP XXX.XXX.XXX.XXX configured on the DNS of app.mydomain.com

I would like to access the 2 child deployment from sub1.app.mydomain.com and sub2.app.mydomain.com

This should be automatic, adding new deployment I cannot change every time the DNS records. Maybe I'm approaching the problem in the wrong way, I'm new in GKE, any suggestions?

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: my-host
spec:
  replicas: 1
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        name: my-host
        type: proxy
    spec:
      hostname: app
      containers:
        - image: nginx:alpine
          name: nginx
          ports:
            - name: nginx
              containerPort: 80
              hostPort: 80
      restartPolicy: Always
status: {}
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: my-subdomain-1
spec:
  replicas: 1
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        name: my-subdomain-1
        type: app
    spec:
      hostname: app
      subdomain: sub1
      containers:
        - image: nginx:alpine
          name: nginx
          ports:
            - name: nginx
              containerPort: 80
              hostPort: 80
      restartPolicy: Always
status: {}
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: my-subdomain-2
spec:
  replicas: 1
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        name: my-subdomain-2
        type: app
    spec:
      hostname: app
      subdomain: sub2
      containers:
        - image: nginx:alpine
          name: nginx
          ports:
            - name: nginx
              containerPort: 80
              hostPort: 80
      restartPolicy: Always
status: {}
---
apiVersion: v1
kind: Service
metadata:
  name: my-expose-dns
spec:
  ports:
    - port: 80
  selector:
    name: my-host
  type: LoadBalancer

Answer 1

SOLVED!

This is the correct nginx configuration:

server {
  listen       80;
  server_name ~^(?<subdomain>.*?)\.;
  resolver kube-dns.kube-system.svc.cluster.local valid=5s;

  location / {
      proxy_pass         http://$subdomain.my-internal-host.default.svc.cluster.local;
      root   /usr/share/nginx/html;
      index  index.html index.htm;
  }

  error_page   500 502 503 504  /50x.html;
  location = /50x.html {
      root   /usr/share/nginx/html;
  }
}

Answer 2

It could be a solution, for my case I need something more dynamic. I would not update the ingress each time I add a subdomain.

I've almost solved using an nginx proxy like this:

    apiVersion: extensions/v1beta1
    kind: Deployment
    metadata:
    name: my-subdomain-1
    spec:
    replicas: 1
    strategy: {}
    template:
        metadata:
        creationTimestamp: null
        labels:
            name: my-subdomain-1
            type: app
        spec:
        hostname: sub1
        subdomain: my-internal-host
        containers:
            - image: nginx:alpine
            name: nginx
            ports:
                - name: nginx
                containerPort: 80
                hostPort: 80
        restartPolicy: Always
    status: {}
    ---
    apiVersion: extensions/v1beta1
    kind: Deployment
    metadata:
    name: my-subdomain-2
    spec:
    replicas: 1
    strategy: {}
    template:
        metadata:
        creationTimestamp: null
        labels:
            name: my-subdomain-2
            type: app
        spec:
        hostname: sub2
        subdomain: my-internal-host
        containers:
            - image: nginx:alpine
            name: nginx
            ports:
                - name: nginx
                containerPort: 80
                hostPort: 80
        restartPolicy: Always
    status: {}
    ---
    apiVersion: v1
    kind: ConfigMap
    metadata:
    name: nginx-config-dns-file
    data:
    nginx.conf: |
        server {
        listen       80;
        server_name ~^(?.*?)\.;

        location / {
            proxy_pass         http://$subdomain.my-internal-host;
            root   /usr/share/nginx/html;
            index  index.html index.htm;
        }

        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   /usr/share/nginx/html;
        }
        }
    ---
    apiVersion: extensions/v1beta1
    kind: Deployment
    metadata:
    name: my-proxy
    spec:
    replicas: 1
    strategy: {}
    template:
        metadata:
        creationTimestamp: null
        labels:
            name: my-proxy
            type: app
        spec:
        subdomain: my-internal-host
        containers:
            - image: nginx:alpine
            name: nginx
            volumeMounts:
                - name: nginx-config-dns-file
                mountPath: /etc/nginx/conf.d/default.conf.test
                subPath: nginx.conf
            ports:
                - name: nginx
                containerPort: 80
                hostPort: 80
        volumes:
            - name: nginx-config-dns-file
            configMap:
                name: nginx-config-dns-file
        restartPolicy: Always
    status: {}
    ---
    apiVersion: v1
    kind: Service
    metadata:
    name: my-internal-host
    spec:
    selector:
        type: app
    clusterIP: None
    ports:
        - name: sk-port
        port: 80
        targetPort: 80
    ---
    apiVersion: v1
    kind: Service
    metadata:
    name: sk-expose-dns
    spec:
    ports:
        - port: 80
    selector:
        name: my-proxy
    type: LoadBalancer

I did understand that I need the service 'my-internal-host' to allow all the deployments to see each other internally. The problem now is only the proxy_pass of nginx, if I change it with 'proxy_pass http://sub1.my-internal-host;' it works, but not with the regexp var.

The problem is related to the nginx resolver.

Answer 3

You want Ingress. There are several options available (Istio, nginx, traefik, etc). I like using nginx and it's really easy to install and work with. Installation steps can be found at kubernetes.github.io.

Once the Ingress Controller is installed, you want to make sure you've exposed it with a Service with type=LoadBalancer. Next, if you are using Google Cloud DNS, set up a wildcard entry for your domain with an A record pointing to the external IP address of your Ingress Controller's Service. In your case, it would be *.app.mydomain.com.

So now all of your traffic to app.mydomain.com is going to that load balancer and being handled by your Ingress Controller, so now you need to add Service and Ingress Entities for any service you want.

apiVersion: v1
kind: Service
metadata:
  name: my-service1
spec:
  selector:
    app: my-app-1
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80
  type: ClusterIP

apiVersion: v1
kind: Service
metadata:
  name: my-service2
spec:
  selector:
    app: my-app2
  ports:
  - protocol: TCP
    port: 80
    targetPort: 80
  type: ClusterIP

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: name-virtual-host-ingress
spec:
  rules:
  - host: sub1.app.mydomain.com
    http:
      paths:
      - backend:
          serviceName: my-service1
          servicePort: 80
  - host: sub2.app.mydomain.com
    http:
      paths:
      - backend:
          serviceName: my-service2
          servicePort: 80

Routing shown is host based, but you just as easily could have handled those services as path based, so all traffic to app.mydomain.com/service1 would go to one of your deployments.