CODEWITHSUNDEEP
kubernetesistionginx-ingressmtlsistio-sidecar
Aleš Krajník
Aleš Krajník

Reputation: 217

Route TCP traffic from ingress-nginx to istio service mesh with mTLS mode STRICT

I am using ingress-nginx v1.3.0 (installed with Helm chart v4.2.3) as a load balancer on DigitalOcean's Kubernetes. In the Kubernetes cluster, I am running istio v1.15.0 service mesh. The mesh runs with peer authentication mTLS mode set to "STRICT" (traffic is required to be properly encrypted).

I was able to successfully connect nginx with istio service mesh for HTTP/HTTPS traffic, but I didn't find a way how to do the same for TCP traffic.

When nginx forwards HTTP traffic, it can either send it directly to the K8s pods (that's the default behaviour), or it can be configured to use the K8s services endpoints using annotations nginx.ingress.kubernetes.io/service-upstream: "true" and nginx.ingress.kubernetes.io/upstream-vhost: "..." on the Ingress object (as documented here).

Without these annotations, istio's Envoy sidecar on the nginx pod does consider the traffic directed to the upstream pod as mesh traffic (it sends the traffic to the PassthroughCluster) and won't encrypt it correctly, so the upstream Envoy sidecar won't accept the traffic due to the mTLS mode set to "STRICT". With the annotations, Envoy recognizes the traffic as mesh traffic and encrypts it correctly.

I am trying to solve the same for TCP traffic. It seems that nginx sends the TCP traffic directly to the pods, same as the default behaviour for HTTP/HTTPS traffic (= when the annotations are not set). However, TCP traffic forwarding is not defined in an Ingress object, so it's not possible to apply the aforementioned annotations. As described above, this results in the TCP traffic to pass unecrypted and the upstream Envoy ending up dropping it.

One solution would be in instructing nginx to send the TCP traffic to the service endpoint similarly as it's possible with HTTP/HTTPS traffic. However, I was not able to find any way how to achieve that.

Perhaps another solution would be telling istio to consider the pod-to-pod traffic (as opposed to the pod-to-service-to-pod traffic) to be mesh traffic too, instructing Envoy to encrypt it too.

Any ideas how to achieve that while keeping the mTLS settings to STRICT? More specifically, having the traffic encrypted is a must for me and thus I can't accept changing the mTLS settings as a solution.

Thank you!

Upvotes: 0

Views: 764

Answers (1)

Fariya Rahmat
Fariya Rahmat

Reputation: 3250

As mentioned in the GitHub link

There is no way to configure the ingress-nginx-controller to do that.

Currently the project is in a feature-freeze phase doing stabilization work so no features will be worked on till end of the 2022 or a even a few months after. You are already aware that the TCP/UDP support opens a port on the controller and you already know that the reverseproxy component is nginx. So I can't even imagine how not only this feature but any layer7 behaviour can be altered/implemented. But I am not a developer so wait for other comments on that aspect.

Upvotes: 0

Related Questions