dotnet application in GKE with nginx ingress controller and proxy

Question

I have a dotnet application running on Kestrel and hosting it in a Linux container on GKE. Alongside the container, I am running a sidecar nginx container to proxy to it. I've read that Kestrel isn't as feature rich thus including the nginx sidecar.

The issue I am having is I either keep getting a 502 or 404 not found. Running local curl requests following redirects does work though.

This returns a proper response from my nginx -> Kestrel

curl -vL "http://127.0.0.1"

Hitting it externally through the public lb,

response 404 (backend NotFound), service rules for [ /index.html ] non-existent
``

This is my nginx.conf

worker_processes 1;

events { worker_connections 1024; }

http {

    sendfile on;

    upstream web-api {
        server 127.0.0.1:5000;
    }

    server {
        listen 80;
        server_name $hostname;

        location /nginx-health {
             return 200 "healthy\n";
        }

        location / {
            proxy_pass         http://web-api;
            proxy_redirect     off;
            proxy_http_version 1.1;
            proxy_cache_bypass $http_upgrade;
            proxy_set_header   Upgrade $http_upgrade;
            proxy_set_header   Connection keep-alive;
            proxy_set_header   Host $host;
            proxy_set_header   X-Real-IP $remote_addr;
            proxy_set_header   X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header   X-Forwarded-Proto $scheme;
            proxy_set_header   X-Forwarded-Host $server_name;
        }
    }
}

My Ingress

Name:             app
Namespace:        app
Address:          34.120.149.155
Default backend:  default-http-backend:80 (<none>)
TLS:
  app-tls terminates external_url
Rules:
  Host                                   Path  Backends
  ----                                   ----  --------
  <external_url>
                                         /   app:80 (10.108.21.149:80)
Annotations:
  certmanager.k8s.io/cluster-issuer:            letsencrypt
  ingress.kubernetes.io/forwarding-rule:        k8s2-fr-6bwo4q66-app-2jv0uft5
  ingress.kubernetes.io/https-forwarding-rule:  k8s2-fs-6bwo4q66-app-2jv0uft5
  ingress.kubernetes.io/https-target-proxy:     k8s2-ts-6bwo4q66-app-2jv0uft5
  ingress.kubernetes.io/target-proxy:           k8s2-tp-6bwo4q66-app-2jv0uft5
  ingress.kubernetes.io/url-map:                k8s2-um-6bwo4q66-app-2jv0uft5
  meta.helm.sh/release-name:                    app
  ingress.kubernetes.io/backends:               {"k8s-be-30587--b22f31f8e3f41440":"HEALTHY","k8s-be-31967--b22f31f8e3f41440":"HEALTHY"}
  ingress.kubernetes.io/ssl-cert:               k8s2-cr-6bwo4q66-rn3hwilrxhwvg79m-506e1c732112861c
  ingress.kubernetes.io/static-ip:              k8s2-fr-6bwo4q66-labs-createstudio-createdataservice-2jv0uft5
  meta.helm.sh/release-namespace:               app

My service

Name:                     app
Namespace:                app
Labels:                   app.kubernetes.io/instance=app
                          app.kubernetes.io/managed-by=Helm
                          app.kubernetes.io/name=app
                          app.kubernetes.io/version=0.1.0
                          helm.sh/chart=app-0.1.0
Annotations:              beta.cloud.google.com/backend-config: {"ports": {"80":"app-config"}}
                          meta.helm.sh/release-name: app
                          meta.helm.sh/release-namespace: app
Selector:                 app.kubernetes.io/instance=app,app.kubernetes.io/name=app
Type:                     NodePort
IP:                       10.181.45.135
Port:                     http  80/TCP
TargetPort:               80/TCP
NodePort:                 http  30587/TCP
Endpoints:                10.108.21.149:80
Session Affinity:         None
External Traffic Policy:  Cluster
Events:                   <none>

I updated all the names/namespaces/urls to something more generic so I do not expose too much information on my app.

I have a feeling its due to the ingress path on the host being just /.

I have also noticed that when hitting nginx externally, I get a 301 redirect which proxies to the Kestrel server. After that Kestrel returns the 301 to nginx and I feel like this is where the loop is. Ie, when Kestrel returns the response, it goes out through the external URL again and sends the request back to nginx from the outside. Hope that makes sense.

Hope anyone could shed some light on this. Cheers!

Answer 1

Since GKE ingress controller can act as a reverse proxy for you (providing SSL termination, for example), there is no need to add an nginx sidecar, and you can route your requests directly to your container application.