Keeping asp.net core config out of your source and your pipelines

Question

I'm working on an asp.net core project and I'm trying to figure out how to keep my source and my pipelines 100% secret free.

I've got a VM running the azure agent and an azure dev ops pipelines for build and release. If i delete the site on the VM, the release pipeline will auto-magically recreate it for me and deploy the latest build.

Super cool.

Now I read up on best practices for configuring a .Net core app and I found this article: https://www.humankode.com/asp-net-core/asp-net-core-configuration-best-practices-for-keeping-secrets-out-of-source-control

So its a bad idea to keep secrets in code, that makes perfect sense. But if i apply the same security principals to Yaml, then surely I shouldn't place secrets in my pipelines either. But I need the pipelines to be able to just recreate the site from scratch and it should just work. Somehow the site needs to know where its default sql connection is, or it needs to have a key to the azure app config service. I shouldn't have to log onto the VM and create an appsettings.json manually after every release!

So whatever the site needs to operate needs to be included in the pipeline, therefore some artifact, or included in the code.

I've googled for days, but I can't seem to find any info on how to fully automate this. I've considered creating a custom configuration provider that reads from the actual VM registry, but that feels wrong too. I basically need a config option that is NOT hosted in the site itself. So i set it up once on the VM and never again.

Answer 1

The approach that Lex Li lists in the comments is the Microsoft recommended way of securing "secrets" in pipelines.

Ben Smith's answer in my opinion is just as good, maybe slightly less secure.

I use this approach in our organization. All of our release pipelines do the final configuration transformation with the appropriate settings based on the environment they are being deployed to.

i.e db connections are transformed at the dev, test and UAT and production deployment stages.

I keep the relevant secrets in the pipeline variables as protected secrets. I do this for 2 reasons:

1. Only a select number of trusted personnel have access to the release pipeline definitions.
2. Even if someone does have access to those definitions - you cannot see a secured variable. Even you you "undo the padlock" on the variable tab - you cannot see what the setting is.

Our actual secrets are then stored in our enterprise secret vault.

Using the Azure Key Vault is definitely a good approach. However we already have a centralized place to keep our stuff; I don't want it in 3 spots.

I would be remiss to not include Variable Groups as part of the pipeline process. Same concept as the build / release variables - the difference is you can now share them in one spot.

These are all opinions of course. This is just one way of doing this; which I feel is a pretty good balance of security and flexibility.

Answer 2

In addition to the suggestions in your questions comments, you can also store secrets in the pipeline "Variables" section.

In here you can add variables and then mark them as secret by selecting "Keep this value secret". Once you've saved a secret its value is then obfuscated i.e. you can make use of it but you can no long see its original value within Azure Devops (which admittedly can be rather frustrating if you want to revisit the variable to check it!).

You can then reference the secret variable in your pipeline YAML using the syntax:

$(variable-name)

So this approach keeps secrets safe within Azure Devops until they need to be resolved by the pipeline YAML script.