Helm charts and Ingress resources

Question

I'm pretty new to K8s.

I'm trying to make my app visible to the outside world. I have deployed this Nginx Ingress Controller with my apps, using Helm and helmfile which has given me an external ip address and a load balancer.

As I understand it, I now need an ingress resource to configure the route mappings.

I plan on using this ingress resource as a starting point.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: minimal-ingress
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  ingressClassName: nginx-example
  rules:
  - http:
      paths:
      - path: /testpath
        pathType: Prefix
        backend:
          service:
            name: test
            port:
              number: 80

My apps are deployed in separate namespaces on port 80.

My question is: where do I put the yaml for creating the resources?

I want to keep everything in Helm if possible to make managing the configuration easier, so I don't want to use kubectl unless i have to.

My helmfile

repositories:
 
- name: stable
  url: https://charts.helm.sh/stable
- name: nginx-stable
  url: https://helm.nginx.com/stable

releases:

  # other apps configured here

  - name: ingress-nginx
    namespace: ingress
    createNamespace: true
    chart: nginx-stable/nginx-ingress
    values:
      - ./ingress/values.yaml
    version: 0.10.4
    installed: true 

My Ingress Controller values.yaml:

---
rbac:
  create: true

serviceAccount:
  create: true
  name: nginx-ingress-public

controller:
  ingressClassResource:
    enabled: true
    default: true

  replicaCount: 3
  minAvailable: 3
  updateStrategy:
    rollingUpdate:
      maxSurge: 3
      maxUnavailable: 0

Answer 1

This is a step by step guide for using Ingress.

Full source code here.

Step 1: Create helm chart

helm create test-app-api

Step 2: Install Ingress controller

This command takes care of installing it. Reference.

helm upgrade --install ingress-nginx ingress-nginx \
  --repo https://kubernetes.github.io/ingress-nginx \
  --namespace ingress-nginx --create-namespace

A pod will be deployed which you can check:

kubectl -n ingress-nginx get pod -o yaml

The information you need from this controller is ingressClassName which you'll put it in your values.yaml file, which will eventually make it to ingress.yaml file.

Find the ingressClassName of your controller by either running this command: kubectl get ingressclasses or finding it through K8s dashboard.

Command way:

Dashboard way:

Step 3: Update ingress section in values.yaml

test-app-api: 
  # Other settings ...
  ingress:
    enabled: true
    className: nginx   # <--- ADD THIS GUY
    annotations:
      # Reference: https://kubernetes.github.io/ingress-nginx/examples/rewrite/
      nginx.ingress.kubernetes.io/use-regex: "true"
      nginx.ingress.kubernetes.io/rewrite-target: /$2
    hosts:
      - host: chart-example.local
        paths:
          - path: /my-test-app(/|$)(.*)
            pathType: ImplementationSpecific
  # Other settings...

Step 4: Deploy

helm upgrade --install test-app-release . \
--namespace=local \
--set test-app-api.image.tag="0.1.0" \
--create-namespace \
--debug \
--dry-run

Do a dry run to inspect the manifest and if everything looks good, run the command without the --dry-run flag.

Step 5: Update hosts file

Check the ingress you deployed to see what address was assigned to your host because you'll be using that address to update your hosts file.

kubectl get ingress -n local

Also seen in controller logs:

W1119 05:14:31.194021       7 controller.go:1214] Service "local/test-app-release-test-app-api" does not have any active Endpoint.
I1119 05:15:19.437846       7 status.go:304] "updating Ingress status" namespace="local" ingress="test-app-release-test-app-api" currentValue=null newValue=[{"hostname":"localhost"}]

Now add this mapping to hosts file. (I'm on a mac and using vim to edit the hosts file.)

sudo vim /etc/hosts

Enter the server IP address at the bottom of the hosts file, followed by a space, and then the domain name.

Save and exit with :wq.

Step 6: Test the app

Now, you should be able to reach the app from outside the cluster.
For eg: http://chart-example.local/my-test-app/weatherforecast

Answer 2

You should deploy the ingress controller only once as it can handle all ingress traffic for your whole cluster.

Sometimes it makes sense to deploy multiple, for example we run 2 ingress controller. 1 for internal traffic (private IP) and one for external traffic (public IP).

Once you have that, you just tell your other helm releases to use its ingress class.

The ingress manifest is usually a template of your helm chart. So you put it in templates. If you do helm create my-app, you get a good starting point, including ingress.

Once you have an ingress template in your chart, you can add some reasonable defaults for this template to the values.yaml of the chart, as usual.

When deploying the chart, you can use certain flags to override the defaults. i.e. -f and --set.