EKS ALB ingress route by port

Question

We have a requirement to make an application available via port. For example http://example.com:8180 and http://example.com:8181 should resolve to the application.
The ports are opened with the listen-ports annotation:

alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS":443}, {"HTTP":8180}, {"HTTP":8181}]'

I tried following rules:

rules:
    - host: example.com
      http:
        paths:
          - path: /*
            backend:
              serviceName: ssl-redirect
              servicePort: use-annotation
          - path: /1.0/*
            backend:
              serviceName: some-server-side-app
              servicePort: 8080
          - path: /*
            backend:
              serviceName: some-webpage
              servicePort: 80
      8180:
        paths:
          - path: /*
            backend:
              serviceName: app-reachable-via-port
              servicePort: 8180
      8181:
        paths:
          - path: /*
            backend:
              serviceName: app-reachable-via-port
              servicePort: 8181

This throws an error because of malformed ingress-configuration

I also found this guide on github for ingress-nginx https://github.com/kubernetes/ingress-nginx/blob/master/docs/user-guide/exposing-tcp-udp-services.md and tried to resemble it but with not luck so far.

Is anyone aware of how this can be achieved with ALB-Ingress?

Answer 1

This thing bugged me a lot. I found a workaround. The solution is to make different ingress for each of the ports. Here's the example:

http://example.com:80  -> Redirect to 443
https://example.com:443  -> Forward to service-3000
http://example.com:8180  -> Forward to service-8180
http://example.com:8181  -> Forward to service-8181

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ssl-redirect
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}]'
    alb.ingress.kubernetes.io/group.name: ingress-group
    alb.ingress.kubernetes.io/actions.ssl-redirect: '{"Type": "redirect", "RedirectConfig": { "Protocol": "HTTPS", "Port": "443", "StatusCode": "HTTP_301"}}'
spec:
  ingressClassName: alb
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: ssl-redirect
            port:
              name: use-annotation

---

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: https-ingress
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    external-dns.alpha.kubernetes.io/hostname: example.com
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
    alb.ingress.kubernetes.io/group.name: ingress-group
spec:
  ingressClassName: alb
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: service-3000
            port:
              number: 3000

---

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: port-8180-rules
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 8180}]'
    alb.ingress.kubernetes.io/group.name: ingress-group
spec:
  ingressClassName: alb
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: service-8180
            port:
              number: 8180

---

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: port-8181-rules
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":8181}]'
    alb.ingress.kubernetes.io/group.name: ingress-group
spec:
  ingressClassName: alb
  rules:
  - host: example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: service-8181
            port:
              number: 8181

For each of the port the ingress rule is different but as I'm using same alb.ingress.kubernetes.io/group.name name, only one ALB will be created. But when I add the SSL redirection annotation (alb.ingress.kubernetes.io/ssl-redirect: '443'), the rules for all the ports have been updated and port 8180 and 8181 also forwarded the request to 443. So I had to add the ssl-redirection rule manually.

(Here AWS Loadbalancer controller version: 1.26)

Answer 2

If I'm understanding what you're looking, you want to differentiate your inbound traffic and send to services based on the port your inbound traffic comes in on?

You might be able to accomplish this with the AWS Load Balancer Controller. I have not tested the object below in this configuration using http-header, but I do use something very similar myself. I think this might be worth testing based on the documentation from AWS here. You'll need to experiment with this though. This example also assumes that each of your services in the cluster are listening on the same port you're using to direct traffic -- this could easily be changed to whatever port your services are actually listening on.

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: example-ingress
  annotations:
    kubernetes.io/ingress.class: alb
    alb.ingress.kubernetes.io/group.name: example
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/security-groups: sg-01234567898765432
    alb.ingress.kubernetes.io/ip-address-type: ipv4
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTP": 80}, {"HTTPS": 443}, {"HTTP": 8180}, {"HTTP": 8181}]'
    
    alb.ingress.kubernetes.io/actions.response-503: >
      {"type":"fixed-response","fixedResponseConfig":{"contentType":"text/plain","statusCode":"503","messageBody":"Unknown Host"}}

    alb.ingress.kubernetes.io/actions.some-webpage: >
      {"type":"forward","forwardConfig":{"targetGroups":[{"serviceName":"some-webpage","servicePort":80,"weight":100}]}}
    alb.ingress.kubernetes.io/conditions.some-webpage: >
      [{"field":"http-header","HttpHeaderConfig":{"HttpHeaderName":"Host","Values":["example.com:80"]}}]

    alb.ingress.kubernetes.io/actions.app1-reachable-via-port: >
      {"type":"forward","forwardConfig":{"targetGroups":[{"serviceName":"app1-reachable-via-port","servicePort":8180,"weight":100}]}}
    alb.ingress.kubernetes.io/conditions.app1-reachable-via-port: >
      [{"field":"http-header","HttpHeaderConfig":{"HttpHeaderName":"Host","Values":["example.com:8180"]}}]

    alb.ingress.kubernetes.io/actions.app2-reachable-via-port: >
      {"type":"forward","forwardConfig":{"targetGroups":[{"serviceName":"app2-reachable-via-port","servicePort":8181,"weight":100}]}}
    alb.ingress.kubernetes.io/conditions.app2-reachable-via-port: >
      [{"field":"http-header","HttpHeaderConfig":{"HttpHeaderName":"Host","Values":["example.com:8181"]}}]

    alb.ingress.kubernetes.io/target-type: instance
    alb.ingress.kubernetes.io/load-balancer-attributes: routing.http2.enabled=true,idle_timeout.timeout_seconds=600
    alb.ingress.kubernetes.io/certificate-arn: arn:aws:acm:us-east-2:999999999999:certificate/11111111-1111-1111-1111-111111111111,arn:aws:acm:us-east-2:999999999999:certificate/22222222-2222-2222-2222-222222222222
    alb.ingress.kubernetes.io/ssl-policy: ELBSecurityPolicy-2016-08
spec:
  backend:
    serviceName: response-503
    servicePort: use-annotation
  rules:
    - http:
        paths:
          - backend:
              serviceName: ssl-redirect
              servicePort: use-annotation
          - backend:
              serviceName: some-webpage
              servicePort: use-annotation
          - backend:
              serviceName: app1-reachable-via-port
              servicePort: use-annotation
          - backend:
              serviceName: app2-reachable-via-port
              servicePort: use-annotation

Another solution might be ALB TargetGroupBinding. You lose some of the advantage of allowing EKS to provision and manage your ALBs and Target Groups on your behalf, but you maintain full control over your ALB and Target Group configurations. With TargetGroupBinding, you still need the AWS Load Balancer Controller in your cluster, but you create the ALB and Target Groups yourself, then use the TargetGroupBinding object to map services from your cluster to specific Target Group ARNs:

apiVersion: elbv2.k8s.aws/v1beta1
kind: TargetGroupBinding
metadata:
  name: demo1-tgb
spec:
  serviceRef:
    name: demo1-service 
    port: 80
  targetGroupARN: arn:aws:elasticloadbalancing:us-east-2:121212121212:targetgroup/my-target-group/cbc9f05b05caea6b

Good luck - please update again once you get it working as required.