CODEWITHSUNDEEP
amazon-web-servicessslistionlb
user3088242
user3088242

Reputation: 11

Istio - Terminate TLS at AWS NLB

I'm using EKS and latest Istio installed via Helm. I'm trying to implement TLS based on a wildcard cert we have for our domain in AWS certificate manager. I'm running into a problem where the connection between the client and the NLB works, with TLS being terminated there, but the NLB can't talk to the istio LB over the secure port. In the AWS console I can rewrite the forwarding rules to forward traffic from port 443 to the standard istio http target, but I can't find a way to do this via code. I'm trying to avoid all click-ops. Here is my Helm overrides for the gateway:

gateways:
  istio-ingressgateway:
    serviceAnnotations:
        service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
        service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "arn:aws:acm:XXXXXXXXXXXXXXXXXX:certificate/XXXXXXXXXXXXXXXXXXXX"
        service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"
        service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"

So what I'm expecting to occur here is:

Client:443 --> NLB:443 --> Istio Gateway:80
but what I end up with is
Client:443 --> NLB:443 --> Istio Gateway:443

Does anyone have any thoughts on how to get this to work via code? Alternately if someone can point me to a resource to get tls communication between the NLB and Istio working I'm good with that too.

Upvotes: 1

Views: 2505

Answers (3)

hoalongnatsu
hoalongnatsu

Reputation: 41

The answer to solving this problem is to switch the target group's TCP protocol to TLS.

Following these steps. NLB annotations:

annotations:
  ...
  service.beta.kubernetes.io/aws-load-balancer-ssl-ports: 'https'
  service.beta.kubernetes.io/aws-load-balancer-ssl-cert: xxx
  ...

Generate a self-signed certificate:

openssl req -x509 -sha256 -nodes -newkey rsa:2048 -subj "/O=devopsvn.tech/CN=devopsvn.tech" -keyout devopsvn.tech.key -out devopsvn.tech.crt

Generate a certificate signing request (CSR):

openssl req -out sub.devopsvn.tech.csr -newkey rsa:2048 -nodes -keyout sub.devopsvn.tech.key -subj "/CN=sub.devopsvn.tech/O=sub.devopsvn.tech"

Sign the CSR with the self-signed certificate:

openssl x509 -req -CA devopsvn.tech.crt -CAkey devopsvn.tech.key -set_serial 0 -in sub.devopsvn.tech.csr -out sub.devopsvn.tech.crt

Create secret:

kubectl create secret tls tls-cert --key=sub.devopsvn.tech.key --cert=sub.devopsvn.tech.crt -n istio-system

Istio Gateway config:

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: istio-gateway
spec:
  selector:
    istio: istio-gateway
  servers:
    - port:
        number: 443
        name: https
        protocol: HTTPS
      tls:
        mode: SIMPLE
        credentialName: tls-cert
      hosts:
        - "*"

Create a new TargetGroup with the TLS protocol

TLS Protocol

Using TargetGroupBinding to bind the a TargetGroup to NLB.

Upvotes: 1

Jay Sithiporn
Jay Sithiporn

Reputation: 91

Your service annotation already correct what is mising is to change istio gateway port 443 to HTTP

apiVersion: networking.istio.io/v1beta1
kind: Gateway
metadata:
  name: http-gateway-external
  namespace: istio-ingress
spec:
  selector:
    istio: gateway-external
  servers:
  - hosts:
    - '*'
    port:
      name: http
      number: 80
      protocol: HTTP
  - hosts:
    - '*'
    port:
      name: https
      number: 443
      protocol: HTTP     # Change from HTTPS to HTTP

Upvotes: 0

Bruno_Ferreira
Bruno_Ferreira

Reputation: 1344

Probably, what is happening is that if you terminate TLS on the load balancer it won't carry SNI to the target group. I had the exact same issue and I ended up solving it by setting the host as '*' on the ingress Gateway and then specifying the hosts on the different VirtualServices (as recommended here and also on istio's official docs).

Upvotes: 1

Related Questions