CODEWITHSUNDEEP
linuxamazon-web-servicesaws-cliaws-secrets-manager
Moddaman
Moddaman

Reputation: 2808

Parsing secrets from AWS secrets manager using AWS cli

I am retrieving secrets I have stored in AWS secrets manager with the AWS cli like this:

aws secretsmanager get-secret-value --secret-id secrets

Which returns

arn:aws:secretsmanager<ID>:secret:my_secrets <number> my_secrets {"API_KEY":"ABCDEFGHI"}       <UUID string>
VERSIONSTAGES   AWSCURRENT

Does anyone know how I only get the secret ("API_KEY": "ABCDEFGHI")? I need to move these secrets to my register-task-definition environment variables. The best way would be to store them in a file and delete it after us or store them in variable. It is running on a linux machine.

Upvotes: 77

Views: 116853

Answers (16)

Thomas Saied
Thomas Saied

Reputation: 202

Just use this command:

aws secretsmanager get-secret-value --secret-id <secrets name> --query SecretString --output text > .env

Upvotes: 1

Wolfgang Kuehn
Wolfgang Kuehn

Reputation: 12936

If you have vector by DataDog installed and do not want to use jq, the VectorRemapLanguage (vrl) is able to do the work. In this bash function we use map_values to replace all JSON fields by there parsed value:

function vq() {
  local data=$1
  local query=$2
  data=${data//$'\n'/}  # Remove newlines
  r=$(($HOME/vector/bin/vector vrl | tail -n 2) << EOF
. = map_values($data) -> |value| {
  struct, err = parse_json(value)
  if err != null {value} else {struct}
}
.$query
EOF
)
  printf '%s' "${r//\"/}"
}

Then retrieve the AWS secret with

data=$(aws secretsmanager get-secret-value --secret-id MyName)
vq "$data" SecretString.API_KEY
# -> ABCDEFGHI

Upvotes: 0

uron83
uron83

Reputation: 1

One liner to list all values in SecretString using PowerShell.

(aws secretsmanager get-secret-value --secret-id secretId | ConvertFrom-Json).SecretString | ConvertFrom-Json

Upvotes: 0

Jon
Jon

Reputation: 1142

If your secret will only have one key/pair value, and you only want the value to be printed out, and you don't want to rely on your system having jq installed first, you can do:

aws secretsmanager get-secret-value --secret-id secrets --query SecretString --output text | cut -d: -f2 | tr -d \"}

Upvotes: 16

Maxamis4
Maxamis4

Reputation: 188

I see many JQ examples but Powershell has a pretty awesome integration with AWS. This is the way I do it in Powershell:

Your JSON value

{"API_KEY":"ABCDEFGHI"}

$aws_secret = Get-SECSecretValue -SecretId my_secrets
$mysecret = $aws_secret.SecretString | ConvertFrom-Json
$myapikey = $mysecret.API_KEY
$newsecret = ConvertTo-SecureString -String $myapikey -AsPlainText -Force

The value from the secret manager is a JSON which Powershell can natively convert into a type of array that you can reference. I convert it back into a secure string under the assumption its a secret and you want to pass it in. The code above should work for you. Let me know if you run into any issues with the code I provided.

Upvotes: 1

Heath Borders
Heath Borders

Reputation: 32157

Lots of answers here depend on jq. If you don't want to install any other dependencies, you can use a python3 one-liner:

aws secretsmanager get-secret-value \
   --output text \
   --query SecretString \
   --secret-id my-secret-name \
| python3 -c 'import json, sys; print(json.load(sys.stdin)["my-secret-key"])'

Based on helloV's answer.

Upvotes: 7

Amit Baranes
Amit Baranes

Reputation: 8152

All answers working but require 3rd party integration ( mainly jq ). the following bash command grabs the relevant Value without any other 3rd party solution -

SECRET_ARN=arn:aws:secretsmanager:eu-west-1:123456:secret:/test
SECRET_KEY=DB_PASSWORD
aws secretsmanager get-secret-value \ 
  --secret-id $SECRET_ARN \ 
  --query SecretString \
  --output text | grep -o '"$SECRET_KEY":"[^"]*' |  grep -o '[^"]*$'

Upvotes: 14

Andrew
Andrew

Reputation: 1167

In the vein of "... without jq" answers, here's one for node users. (requires modern bash and nodejs, could easily be rewritten to just use sh by using an echo | instead of the cleaner <<<)

SECRET_ARN="..."
REGION=us-east-1

SECRET_BLOB=$(aws secretsmanager get-secret-value --region="$REGION" --output=text --query SecretString --secret-id "$SECRET_ARN")

MY_VALUE=$(node -pe 'JSON.parse(require("fs").readFileSync("/dev/stdin").toString()).myKey' <<< "$SECRET_BLOB")

MY_OTHER_VALUE=$(node -pe 'JSON.parse(require("fs").readFileSync("/dev/stdin").toString()).myOtherKey' <<< "$SECRET_BLOB")

If you need to pull multiple values from the secret, you'll want to cache the json blob in an env var. If you only need a single value though:

MY_VALUE=$(aws secretsmanager get-secret-value --region="$REGION" --output=text --query SecretString --secret-id "$SECRET_ARN" | node -pe 'JSON.parse(require("fs").readFileSync("/dev/stdin").toString()).myKey' <<< "$SECRET_BLOB")

Upvotes: 1

Ashish Karpe
Ashish Karpe

Reputation: 5804

Script to List all available AWS secrets to a /tmp/name.text and find specific secret values from it

Note needs AWS CLI configure to run this script successfully

#!/bin/bash

aws secretsmanager list-secrets | grep  "Name" | awk '{print $2}' | tr -d '"' | sed 's/,/ /g' > /tmp/name.text

for line in `cat /tmp/name.text`
do

echo $line >> /tmp/secrets-values.txt

aws secretsmanager get-secret-value --secret-id "$line" | grep "XYZ" >>  /tmp/secrets-values.txt
done

Upvotes: 1

TanishGuleria
TanishGuleria

Reputation: 11

PowerShell solution without Jq

$a = aws secretsmanager get-secret-value --region <region> --secret-id <secret-name>  | ConvertFrom-Json

$a all json converted to objects type

Output

ARN           : xxxxxx
Name          : postgxxx
VersionId     : fxxxx-xx-x-xx
SecretString  : {"key":"value","key2":"value"}
VersionStages : {xxxxx}
CreatedDate   : xxxxx.xx

$b = $a.SecretString | ConvertFrom-Json

Output

key : value
key2 : value

$b.key

**Output** 
value

Upvotes: 1

Chart96
Chart96

Reputation: 480

Use this to get just the value of the secret key. Make sure to fill in your secert ID and the key of the secret:

aws secretsmanager get-secret-value --secret-id <yourSecretID> | jq '.SecretString' | tail -c +2 | head -c -2 | tr -d '\' | jq .<YourSecretKey>

Upvotes: -3

helloV
helloV

Reputation: 52433

Use the --query option of the CLI to extract just the secret.

aws secretsmanager get-secret-value --secret-id secrets --query SecretString --output text

Upvotes: 147

kadir
kadir

Reputation: 1417

Small addition to helloV answer. You can add the output parameter text to remove the quotes.

aws secretsmanager get-secret-value \
   --secret-id secrets \
   --query SecretString \
   --output text

Upvotes: 30

Aseem Jain
Aseem Jain

Reputation: 351

When you have multiple secret and you get json return, you can use get the exact value of password by using 

aws secretsmanager get-secret-value --secret-id <secret_bucket_name> | jq --raw-output '.SecretString' | jq -r .key_for_password

Upvotes: 11

thecloudguy
thecloudguy

Reputation: 351

So I faced a bit of trouble in extracting what I needed, the value for my two variables that I stored in SecretsManager. So here is what worked for me.

NOTE: It's an example from the AWS SecretsManager doc.

I ran this

aws secretsmanager get-secret-value --secret-id MyTestDatabaseSecret --version-stage AWSPREVIOUS

The response of this query is:

{
  "ARN": "arn:aws:secretsmanager:us-west-2:123456789012:secret:MyTestDatabaseSecret-a1b2c3",
  "Name": "MyTestDatabaseSecret",
  "VersionId": "EXAMPLE1-90ab-cdef-fedc-ba987EXAMPLE",
  "SecretString": "{\n  \"username\":\"david\",\n  \"password\":\"BnQw&XDWgaEeT9XGTT29\"\n}\n",
  "VersionStages": [
    "AWSPREVIOUS"
  ],
  "CreatedDate": 1523477145.713
}

Now I want to get the value of username or password to be precise

aws secretsmanager get-secret-value --secret-id MyTestDatabaseSecret --version-stage AWSPREVIOUS | jq --raw-output .SecretString | jq -r ."password"

Output

BnQw&XDWgaEeT9XGTT29

Upvotes: 12

Vijay Kumar
Vijay Kumar

Reputation: 589

aws secretsmanager get-secret-value --secret-id secrets| jq --raw-output '.SecretString' | jq -r .API_KEY

using jq you can print.

Upvotes: 47

Related Questions