How can I easily set up HTTPS between ASP.NET Core Pods inside a Kubernetes cluster?

Question

Assume an application consisting of different web services implemented with ASP.NET Core, each of which is deployed as a Pod in a Kubernetes cluster (AKS to be exact). Now, suppose I want to secure the cluster-internal communication between those services via HTTPS. This requires me to:

1. get TLS certificates for each of the services,
2. have the Pods trust those TLS certificates (or, rather, the signing CA), and
3. rotate the certificates when their validity period ends.

What I've already learned:

• This StackOverflow answer indicates that this adds a lot of complexity and discourages going that route. Nevertheless, I'd like to know what such a setup would comprise.
• Projects such as LettuceEncrypt allow to automate most of the steps 1 and 3 above. You only need a CA that implements the ACME protocol.
• The Kubernetes docs ("Managing TLS in a cluster") mention a Kubernetes API which uses a "protocol that is similar to the ACME draft" to manage CSRs.
• However, in the docs, they're doing all the work manually (setting up a local CA, issuing CSRs manually, signing the CSRs manually using the local CA, all via the cfssl tools) that I'm wondering why on earth I would actually want to use those APIs. What are they doing for me besides storing CSRs as Kubernetes resources?
• The docs also mention that Kubernetes clusters already include a root CA that one could use for the purpose of issuing TLS certificates for Pods, but don't explain how one would do so: "It is possible to configure your cluster to use the cluster root CA for this purpose, but you should never rely on this."
• The quote above seems to suggest and warn against using the cluster root CA at the same time. Why the warning, wouldn't it simplify things a lot if we could use an existing CA?

In my mind, this could be really simple: Just set up Kestrel with LettuceEncrypt, configure it against the cluster root CA, and have all the Pods trust that CA (by importing the corresponding certificate as a trusted root).

Is it that simple? Or what am I missing?

Update 2022-07-26: Note that I need to support Windows containers.

Answer 1

For this purpose you should use mTLS. To archive this with an AKS Cluster you can easily active the Open Service Mesh Add-On. With OSM enabled, you can now encrypt communications between service endpoints deployed in the cluster. The cool thing is the the OSM Add-on integrates with Azure Monitor.

Here an example to do mTLS with ingress-nginx :

To proxy connections to HTTPS backends, we will configure the Ingress and IngressBackend configurations to use https as the backend protocol, and have OSM issue a certificate that Nginx will use as the client certificate to proxy HTTPS connections to TLS backends. The client certificate and CA certificate will be stored in a Kubernetes secret that Nginx will use to authenticate service mesh backends.