Load certificate failing in windows docker container in staging AKS cluster. Same piece of code works in dev aks cluster

Question

we have a C# code to load x509 certs from keyvault. This piece of code worked in our dev environments without any issues. Code here

var signingCertificate = new X509Certificate2(buffer, certificatePassword, X509KeyStorageFlags.Exportable);

Now the same piece of code when deployed to our staging environment fails with below error.

We have checked the aks version and the underlying windows version. Both are identical. We checked the deployment yaml. no change there too.

We were able to change the X509KeyStorageFlags options and get it working. But would like to understand how and why the same code works in lower env and not in our staging deployment.

Any guidance on how to solve this strange issue?

The system cannot find the file specified.    
at System.Security.Cryptography.CryptographicException.ThrowCryptographicException(Int32 hr)    
at System.Security.Cryptography.X509Certificates.X509Utils._LoadCertFromBlob(Byte[] rawData, IntPtr password, UInt32 dwFlags, Boolean persistKeySet, SafeCertContextHandle& pCertCtx)
at System.Security.Cryptography.X509Certificates.X509Utils.LoadCertFromBlob(Byte[] rawData, IntPtr password, UInt32 dwFlags, Boolean persistKeySet, SafeCertContextHandle pCertCtx)    
at System.Security.Cryptography.X509Certificates.X509Certificate.LoadCertificateFromBlob(Byte[] rawData, Object password, X509KeyStorageFlags keyStorageFlags, Boolean passwordProvided)    
at System.Security.Cryptography.X509Certificates.X509Certificate2..ctor(Byte[] rawData, String password, X509KeyStorageFlags keyStorageFlags)

Answer 1

In order to load X509 certificate in a .NET Core application running in aks and assuming you have your aks, acr and akv ready, you can follow the below steps.

Fyi this example is with self-signed cert.

Update your program.cs file as below-

using System;
using System.Security.Cryptography.X509Certificates;

namespace LoadCertificateApp
{
    class Program
    {
        static void Main(string[] args)
        {
            try
            {
                string certificatePassword = Environment.GetEnvironmentVariable("CERTIFICATE_PASSWORD");
                byte[] buffer = Convert.FromBase64String(Environment.GetEnvironmentVariable("CERTIFICATE_BASE64"));
                var signingCertificate = new X509Certificate2(buffer, certificatePassword, X509KeyStorageFlags.MachineKeySet);
                Console.WriteLine("Certificate loaded successfully.");
            }
            catch (Exception ex)
            {
                Console.WriteLine($"Error loading certificate: {ex.Message}");
            }
        }
    }
}

Sample Dockerfile

FROM mcr.microsoft.com/dotnet/runtime:5.0
WORKDIR /app
COPY . .
ENTRYPOINT ["dotnet", "LoadCertificateApp.dll"]

built and pushed-

FROM mcr.microsoft.com/dotnet/aspnet:7.0 AS base
WORKDIR /app
EXPOSE 80

FROM mcr.microsoft.com/dotnet/sdk:7.0 AS build
WORKDIR /src
COPY ["LoadCertificateApp/LoadCertificateApp.csproj", "LoadCertificateApp/"]
RUN dotnet restore "LoadCertificateApp/LoadCertificateApp.csproj"
COPY . .
WORKDIR "/src/LoadCertificateApp"
RUN dotnet build "LoadCertificateApp.csproj" -c Release -o /app/build

FROM build AS publish
RUN dotnet publish "LoadCertificateApp.csproj" -c Release -o /app/publish

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "LoadCertificateApp.dll"]

Create a secret for your k8s deployment-

az acr login --name arkoacr2
kubectl create secret docker-registry acr-secret --docker-server=arkoacr2.azurecr.io --docker-username=<username> --docker-password=<password> --docker-email=<email>
kubectl create secret generic cert-secret --from-literal=CERTIFICATE_PASSWORD=<cert-password> --from-literal=CERTIFICATE_BASE64=<base64-encoded-cert>

you can get this base64 converted value required in the above command, from using the below command-

az keyvault secret download --vault-name arkokv1 --name MyCertificate --file MyCertificate.pfx

certutil -encode MyCertificate.pfx MyCertificate.base64

This above command will convert your cert to base64 file

And accordingly edit your particular application deployment to hold that base64 cert.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: load-certificate-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: load-certificate-app
  template:
    metadata:
      labels:
        app: load-certificate-app
    spec:
      containers:
      - name: load-certificate-app
        image: arkoacr2.azurecr.io/load-certificate-app:latest
        env:
        - name: CERTIFICATE_PASSWORD
          valueFrom:
            secretKeyRef:
              name: cert-secret
              key: CERTIFICATE_PASSWORD
        - name: CERTIFICATE_BASE64
          valueFrom:
            secretKeyRef:
              name: cert-secret
              key: CERTIFICATE_BASE64
      imagePullSecrets:
      - name: acr-secret

Similar issue

Similar issue