CODEWITHSUNDEEP
kubernetesaws-secrets-manager
jonny
jonny

Reputation: 646

Env variable from AWS Secrets Manager in Kubernetes

In EKS I am trying to use SecretProviderClass to provide secrets as environment variables to containers. I can see the secret mounted inside the container but no combination of key/names is allowing me to use it as an environment variable. Insode the container I can cat /mnt/secrets-store/awscredentials And see the output:

{"accesskey":"ABCDE12345","secretkey":"a/long/redacted5tring"}

My SecretProviderClass is below

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: aws-secrets
  namespace: default
spec:
  provider: aws
  parameters:
    objects: |
        - objectName: "path/to/service/awscredentials"
          objectType: secretsmanager
          objectAlias: awscredentials
  secretObjects:
  - secretName: awscredentials
    type: Opaque
    data: 
    - objectName: accesskeyalias
      key: accesskey
    - objectName: secretkeyalias
      key: secretkey

and my deployment:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myservice
  labels:
    team: devops
spec:
  replicas: 1
  selector:
    matchLabels:
      app: myservice
  template:
    metadata:
      labels:
        app: myservice
    spec:
      serviceAccountName: myservice
      volumes:
      - name: secrets-store-inline
        csi:
          driver: secrets-store.csi.k8s.io
          readOnly: true
          volumeAttributes:
            secretProviderClass: "aws-secrets"
      containers:
      - name: myservice
        image: someimage:2
        volumeMounts:
        - name: secrets-store-inline
          mountPath: "/mnt/secrets-store"
          readOnly: true
        env:
        - name: AWS_ACCESS_KEY
          valueFrom:
            secretKeyRef:
              name: awscredentials
              key: accesskey

When I run the deployment without reference to the SecretKeyRef the container runs and I can see the secret under /mnt/secrets-store/awscredentials. However, trying to set the environment variable results in the pod stuck in Pending state and the message: Error: secret "awscredentials" not found I reckon I have mixed up the name and keys somewhere but I've spent hours trying every combination I can think of. What am I missing?

Upvotes: 9

Views: 13128

Answers (4)

Zhongxia Zhou
Zhongxia Zhou

Reputation: 128

Apart from setting syncSecret.enabled=true, I also found in my case that the objectAlias seems to be necessary as well. For example, if I have a secret stored in secrets manager with name as development/api_key with value as {"key": "<some_api_key>"}, then the SecretProviderClass config may look something like:

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: my-secret-provider-class
  namespace: my-namespace
spec:
  provider: aws
  parameters:
    objects: |
      - objectName: development/api_key
        objectType: secretsmanager
        objectAlias: api-key-alias   # This is needed to make a k8s Secret resource created under the namespace. 
  secretObjects:
    - secretName: api-key
      type: Opaque
      data:
        - objectName: api-key-alias
          key: key

After applying the config, I can see the secret by running kubectl -n <namespace> get secrets

Upvotes: 1

Jason Wambach
Jason Wambach

Reputation: 11

For anyone else that still had issues with jmesPath, know that if your keys contain dashes "-" they need to be quoted with single quotes ' in order to work. In my case, I eliminated dashes from my keys and it all started working.

Upvotes: 1

Surya yaramada
Surya yaramada

Reputation: 1

Can you provide example how you got this working

instead 1) mounted the JSON file into the pod and 2) customized the entrypoint script to first parse and source all the variables from the JSON file as bash environment variables

@martin

Upvotes: 0

jonny
jonny

Reputation: 646

I eventually got this sorted. I had followed the AWS documentation for installing the driver which included using a helm chart i.e.

helm install -n kube-system csi-secrets-store secrets-store-csi-driver/secrets-store-csi-driver

However, the documentation failed to point out that with this helm chart you need to specifically set a value syncSecret.enabled=true - as in the code below.

helm install -n kube-system csi-secrets-store secrets-store-csi-driver/secrets-store-csi-driver --set syncSecret.enabled=true

I used helm to uninstall the secrets-store-csi-driver, then re-installed with syncSecret.enabled=true and immediately my secretsmanager secret was available via kubectl get secrets -n default. So if you can see the secrets inside the container in the mounted volume but you can't set them as environment variables you should check that you installed with this value so that the k8s secret is created. Otherwise the secretObjects > secretName section is not actioned.

Upvotes: 12

Related Questions