CODEWITHSUNDEEP
kubernetesgoogle-cloud-platformkubernetes-secretskustomizegoogle-secret-manager
Mr. Developerdude
Mr. Developerdude

Reputation: 9668

Look up secrets from gcloud secrets manager directly as secretGenerator with kustomize

I am setting up my Kubernetes cluster using kubectl -k (kustomize). Like any other such arrangement, I depend on some secrets during deployment. The route I want go is to use the secretGenerator feature of kustomize to fetch my secrets from files or environment variables.

However managing said files or environment variables in a secure and portable manner has shown itself to be a challenge. Especially since I have 3 separate namespaces for test, stage and production, each requiring a different set of secrets.

So I thought surely there must be a way for me to manage the secrets in my cloud provider's official way (google cloud platform - secret manager).

So how would the secretGenerator that accesses secrets stored in the secret manager look like?

My naive guess would be something like this:

secretGenerator:
 - name: juicy-environment-config
   google-secret-resource-id: projects/133713371337/secrets/juicy-test-secret/versions/1
   type: some-google-specific-type

Upvotes: 5

Views: 1751

Answers (3)

Paolo
Paolo

Reputation: 26084

While technically scheduled for deprecation, with kustomize v5.4.1 you can still leverage an exec plugin for this. Here's an example.

Layout:

.
├── base
│   └── secrets
│       ├── kustomization.yaml
│       └── secretGenerator.yaml
└── plugin
    └── util
        └── gcloudsecretgenerator
            └── gcloudSecretGenerator

base/secrets/kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

generators:
- secretGenerator.yaml

base/secrets/secretGenerator.yaml

apiVersion: util
kind: gcloudSecretGenerator
metadata:
  name: whatever
argsOneLiner: |-
  secret1 gcloudSecret1Name
  secret2 gcloudSecret2Name

plugin/util/gcloudsecretgenerator/gcloudSecretGenerator

#!/bin/bash

# Discard the first argument
secret_data=("${@:2}")

# Loop over input arguments and for every other input look up the secret value
for ((i=1;i< ${#secret_data[@]} ;i+=2));
do
  # Get secret and convert to base64
  secret_value=$(gcloud secrets versions access latest --secret=${secret_data[$i]} | base64)
  secret_data[$i]="$secret_value"
done

# Format secret data properly for interpolation
v=$(printf '"%s" : "%s"\n  ' "${secret_data[@]}")

echo "
kind: Secret
apiVersion: v1
metadata:
  name: secret
data:
  $v
"

The path of this plugin is important. The first folder must match the apiVersion (so in this case util), the second folder must be the lower case kind (so in this case gcloudsecretgenerator) and the filename must match the kind (so in this case gcloudSecretGenerator). Make sure the plugin is executable (chmod +x)

This plugin parses the arguments that it receives from argsOneLiner and for every other argument it looks up the secret value from gcloud using an api call.

To run a build we must use KUSTOMIZE_PLUGIN_HOME to tell kustomize where to find the plugin:

$ KUSTOMIZE_PLUGIN_HOME="$(pwd)/plugin" kustomize build base/secrets/ --enable-alpha-plugins
apiVersion: v1
data:
  secret1: ippZ0USQwwTUfptm....
  secret2: ippZ0USQwwTUfptm....
kind: Secret
metadata:
  name: secret

Reference: https://kubectl.docs.kubernetes.io/guides/extending_kustomize/exec_plugins/

Upvotes: 0

damick
damick

Reputation: 1150

There is a Go plugin for this (I helped write it), but plugins weren't supported until more recent versions of Kustomize, so you'll need to install Kustomize directly and run it like kustomize build <path> | kubectl apply -f - rather than kubectl -k. This is a good idea anyway IMO since there are a lot of other useful features in newer versions of Kustomize than the one that's built into kubectl.

As seen in the examples, after you've installed the plugin (or you can run it within Docker, see readme) you can define files like the following and commit them to version control:

my-secret.yaml

apiVersion: crd.forgecloud.com/v1
kind: EncryptedSecret
metadata:
  name: my-secrets
  namespace: default
source: GCP
gcpProjectID: my-gcp-project-id
keys:
- creds.json
- ca.crt

In your kustomization.yaml you would add

generators:
- my-secret.yaml

and when you run kustomize build it'll automatically retrive your secret values from Google Secret Manager and output Kubernetes secret objects.

Upvotes: 1

coderanger
coderanger

Reputation: 54221

I'm not aware of a plugin for that. The plugin system in Kustomize is somewhat new (added about 6 months ago) so there aren't a ton in the wild so far, and Secrets Manager is only a few weeks old. You can find docs at https://github.com/kubernetes-sigs/kustomize/tree/master/docs/plugins for writing one though. That links to a few Go plugins for secrets management so you can probably take one of those and rework it to the GCP API.

Upvotes: 2

Related Questions