CODEWITHSUNDEEP
kuberneteskubernetes-secretskustomizeskaffoldmozilla-sops
Robert Kajic
Robert Kajic

Reputation: 9077

Skaffold before and after deploy hooks when decrypting secrets in before hook and using kustomize

According to the Skaffold lifecycle hooks documentation, I have added the following before and after hooks to my skaffold.yaml:

deploy:
  kubectl:
    hooks:
      before:
        - host:
            command: ["secrets", "--decrypt"]
            os: [darwin, linux]
      after:
        - host:
            command: ["secrets", "--restore"]
            os: [darwin, linux]\

The problem is, even though the hook seems to run at the right time, and even though I can see in my secrets that they were successfully decrypted, Skaffold seems to be passing the encrypted version of my secrets to k8s:

Starting deploy...
Starting pre-deploy hooks...
/home/foo/bar/bin/crypt-age: Performing --decrypt operation on .age-key with GPG Key ID foo...
gpg: encrypted with 4096-bit RSA key, ID foo, created 2024-10-04
      "Foo Bar (setup-generated) <[email protected]>"
Running --decrypt --in-place on secret files in deployment/k8s/base...
/home/foo/bar/bin/crypt-age: Performing --encrypt operation on .age-key with GPG Key ID foo...
Operation completed successfully.
Completed pre-deploy hooks
 - gateway.gateway.networking.k8s.io/gateway unchanged
 - namespace/foo-dev unchanged
 - service/foo-service configured
 - deployment.apps/foo-deployment configured
 - httproute.gateway.networking.k8s.io/foo-http-route configured
 - cluster.postgresql.cnpg.io/foo-postgres-cluster configured
 - The request is invalid: patch: Invalid value: "map[metadata:map[annotations:map[kubectl.kubernetes.io/last-applied-configuration:{\"apiVersion\":\"v1\",\"kind\":\"Secret\",\"metadata\":{\"annotations\":{},\"name\":\"foo-postgres-secret\",\"namespace\":\"foo-dev\"},\"sops\":...

I might add that I'm also instructing Skaffold to use kustomize to apply my configuration:

manifests:
  kustomize:
    paths:
      - deployment/k8s/shared
      - deployment/k8s/overlays/dev

I note that the deploy hooks are under a kubectl section, which does give me pause, but it seems to be the only option there (putting them under a kustomize section is a syntax error).

Finally, if I remove the before hook and decrypt myself before running Skaffold, everything works as expected:

secrets --decrypt && skaffold dev -f deployment/skaffold.yaml --cleanup=false

Is there no way to make this work with Skaffold's built in hooks (while using kustomize, if this is really the issue)?

Upvotes: 0

Views: 154

Answers (0)

Related Questions