CODEWITHSUNDEEP
azuresslload-balancingazure-virtual-machineazure-load-balancer
Tachyon
Tachyon

Reputation: 2411

SSL on .NET Core VMS behind load balancer

I am currently setting up a High Availability (HA) environment with two Azure Virtual Machines running Ubuntu sitting behind a standard Azure load balancer. Now I know the standard load balancer is only layer 4 which means it cannot do SSL offloading.

The two VM's are both running a .NET Core Web API. They would obviously each need the SSL certificate to handle SSL connections coming in from the load balancer.

I know I can purchase an SSL certificate and just setup Kestrel to use the certificate on the Web API itself but I would like the free certificate. I know the other option is to generate the certificate using an nginx server then copy across the certificates to the Web API but this means I would need to repeat the process every 3 months which is quite a ball ache as it means I would have downtime while I take the HA cluster offline to renew the certificate.

Does anyone know of a way to use Lets Encrypt on the two VMs sitting behind the load balancer?

Upvotes: 0

Views: 277

Answers (1)

Tachyon
Tachyon

Reputation: 2411

Preface

Okay so I came right with the above. It required me to write a utility that auto renews my Lets Encrypt certificates using DNS verification. It is quite important that it uses Azure DNS or another DNS provider that has an API as you will need to be able to modify your DNS records directly with either an API or some other interface with your provider.

I am using Azure DNS and it is managing the entire domain for me so the code below is for Azure DNS but you can modify the API to work with any provider of your choosing that has some sort of API.

The second part of this, is not to have any downtime in my high availability (HA) cluster. So what I have done is, is to write the certificate to the database and then read it dynamically on startup of my VM's. So basically every time Kestrel starts it reads the certificate from the DB and then uses that.

Code

Database Model

You will need to add the following model to your database so that you can store the actual certificate particulars somewhere.

public class Certificate
{
    [Key]
    [DatabaseGenerated(DatabaseGeneratedOption.Identity)]
    public long Id { get; set; }
    public string FullChainPem { get; set; }
    public string CertificatePfx { get; set; }
    public string CertificatePassword { get; set; }
    public DateTime CertificateExpiry { get; set; }
    public DateTime? CreatedAt { get; set; }
    public DateTime? UpdatedAt { get; set; }
}

Once you have created the model you will need to place it in your context as follows:

public DbSet<Certificate> Certificates { get; set; }

Application server(s)

On your application servers you would want to use Kestrel to act as the web server and then load the certificate dynamically from the database. So add the following to your CreateWebHostBuilder method. It is important that this is after .UseStartup<Startup>()

.UseKestrel(opt = >{
    //Get the application services
    var applicationServices = opt.ApplicationServices;
    //Create and use scope
    using(var scope = applicationServices.CreateScope()) {
        //Get the database context to work with
        var context = scope.ServiceProvider.GetService < DBContext > ();

        //Get the certificate
        var certificate = context.Certificates.Last();
        var pfxBytes = Convert.FromBase64String(certificate.CertificatePfx);
        var pfxPassword = certificate.CertificatePassword;

        //Create the certificate
        var cert = new X509Certificate2(pfxBytes, pfxPassword);

        //Listen on the specified IP and port
        opt.Listen(IPAddress.Any, 443, listenOpts = >{
            //Use HTTPS
            listenOpts.UseHttps(cert);
        });
    }
});

Lets encrypt utility

So this is the meat of the solution. It handles the certificate requests, challenges, DNS verification and then the storage of the certificates. It also will auto restart each VM instance in Azure that is using the certificates so that they pull the new certificates.

The Main logic is as follows, it will check whether the certificates need to be renewed or not.

static void Main(string[] args) {
    while (true) {
        //Get the latest certificate in the DB for the servers
        var lastCertificate = _db.Certificates.LastOrDefault();

        //Check if the expiry date of last certificate is more than a month away
        if (lastCertificate != null && (lastCertificate.CertificateExpiry - DateTime.Now).TotalDays > 31) {
            //Log out some info
            Console.WriteLine($ "[{DateTime.Now}] - Certificate still valid, sleeping for a day.");
            //Sleep the thread
            Thread.Sleep(TimeSpan.FromDays(1));
        }
        else {
            //Renew the certificates
            RenewCertificates();
        }
    }
}

Okay so this is a lot to go through but it is actually quite simple if you break it down

  1. Create an account
  2. Get the account key
  3. Create a new order for the domain(s)
  4. Loop through all the organizations
  5. Perform DNS validation on each of them
  6. Generate certificates
  7. Save certificates to the DB
  8. Restart the VMs

The actual RenewCertificates method is as follows:

/// <summary>
/// Method that will renew the domain certificates and update the database with them
/// </summary>
public static void RenewCertificates() {
    Console.WriteLine($ "[{DateTime.Now}] - Starting certificate renewal.");
    //Instantiate variables
    AcmeContext acme;
    IAccountContext account;

    //Try and get the setting value for ACME Key
    var acmeKey = _db.Settings.FirstOrDefault(s = >s.Key == "ACME");

    //Check if acme key is null
    if (acmeKey == null) {
        //Set the ACME servers to use
    #if DEBUG
         acme = new AcmeContext(WellKnownServers.LetsEncryptStagingV2);
    #else 
         acme = new AcmeContext(WellKnownServers.LetsEncryptV2);
    #endif
        //Create the new account
        account = acme.NewAccount("[email protected]", true).Result;
        //Save the key to the DB to be used
        _db.Settings.Add(new Setting {
            Key = "ACME",
            Value = acme.AccountKey.ToPem()
        });
        //Save DB changes
        _db.SaveChanges();
    }
    else {
        //Get the account key from PEM
        var accountKey = KeyFactory.FromPem(acmeKey.Value);

        //Set the ACME servers to use
    #if DEBUG 
             acme = new AcmeContext(WellKnownServers.LetsEncryptStagingV2, accountKey);
    #else 
             acme = new AcmeContext(WellKnownServers.LetsEncryptV2, accountKey);
    #endif
        //Get the actual account
        account = acme.Account().Result;
    }

    //Create an order for wildcard domain and normal domain
    var order = acme.NewOrder(new[] {
        "*.yourdomain.tld",
        "yourdomain.tld"
    }).Result;

    //Generate the challenges for the domains
    var authorizations = order.Authorizations().Result;

    //Error flag
    var hasFailed = false;

    foreach(var authorization in authorizations) {
        //Get the DNS challenge for the authorization
        var dnsChallenge = authorization.Dns().Result;
        //Get the DNS TXT
        var dnsTxt = acme.AccountKey.DnsTxt(dnsChallenge.Token);

        Console.WriteLine($ "[{DateTime.Now}] - Received DNS challenge data.");

        //Set the DNS record
        Azure.SetAcmeTxtRecord(dnsTxt);

        Console.WriteLine($ "[{DateTime.Now}] - Updated DNS challenge data.");
        Console.WriteLine($ "[{DateTime.Now}] - Waiting 1 minute before checking status.");

        dnsChallenge.Validate();

        //Wait 1 minute
        Thread.Sleep(TimeSpan.FromMinutes(1));

        //Check the DNS challenge
        var valid = dnsChallenge.Validate().Result;

        //If the verification fails set failed flag
        if (valid.Status != ChallengeStatus.Valid) hasFailed = true;
    }

    //Check whether challenges failed
    if (hasFailed) {
        Console.WriteLine($ "[{DateTime.Now}] - DNS challenge(s) failed, retrying.");
        //Recurse
        RenewCertificates();
        return;
    }
    else {
        Console.WriteLine($ "[{DateTime.Now}] - DNS challenge(s) successful.");

        //Generate a private key
        var privateKey = KeyFactory.NewKey(KeyAlgorithm.ES256);

        //Generate certificate
        var cert = order.Generate(new CsrInfo {
            CountryName = "ZA",
            State = "Gauteng",
            Locality = "Pretoria",
            Organization = "Your Organization",
            OrganizationUnit = "Production",
        },
        privateKey).Result;

        Console.WriteLine($ "[{DateTime.Now}] - Certificate generated successfully.");

        //Get the full chain
        var fullChain = cert.ToPem();

        //Generate password
        var pass = Guid.NewGuid().ToString();

        //Export the pfx
        var pfxBuilder = cert.ToPfx(privateKey);
        var pfx = pfxBuilder.Build("yourdomain.tld", pass);

        //Create database entry
        _db.Certificates.Add(new Certificate {
            FullChainPem = fullChain,
            CertificatePfx = Convert.ToBase64String(pfx),
            CertificatePassword = pass,
            CertificateExpiry = DateTime.Now.AddMonths(2)
        });

        //Save changes
        _db.SaveChanges();

        Console.WriteLine($ "[{DateTime.Now}] - Database updated with new certificate.");

        Console.WriteLine($ "[{DateTime.Now}] - Restarting VMs.");

        //Restart the VMS
        Azure.RestartAllVms();
    }
}

Azure integration

Wherever I called Azure you would need to write your API wrapper to set DNS TXT records, and then the ability to restart the VMs from your hosting provider. Mine was all with Azure so it was pretty simple to do. Here is the Azure code:

/// <summary>
/// Method that will set the TXT record value of the ACME challenge
/// </summary>
/// <param name="txtValue">Value for the TXT record</param>
/// <returns>Whether call was successful or not</returns>
public static bool SetAcmeTxtRecord(string txtValue) {
    //Set the zone endpoint
    const string url = "https://management.azure.com/subscriptions/{subId}/resourceGroups/{resourceGroup}/providers/Microsoft.Network/dnsZones/{dnsZone}/txt/_acme-challenge?api-version=2018-03-01-preview";

    //Authenticate API
    AuthenticateApi();

    //Build up the body to put
    var body = $ "{{\"properties\": {{\"metadata\": {{}},\"TTL\": 225,\"TXTRecords\": [{{\"value\": [\"{txtValue}\"]}}]}}}}";

    //Build up the string content
    var content = new StringContent(body, Encoding.UTF8, "application/json");

    //Create the response
    var response = client.PutAsync(url, content).Result;

    //Return the response
    return response.IsSuccessStatusCode;
}

I hope this is able to help someone else that was in the same predicament as myself.

Upvotes: 2

Related Questions