CODEWITHSUNDEEP
pythonamazon-web-servicessubprocessaws-cli
reuben
reuben

Reputation: 91

Return results from awscli python

Hi I have a aws command which when run in the terminal outputs a list of datasets in AWS quicksight with their storage to a file:

aws quicksight list-data-sets --aws-account-id <foo-id> --region <foo-region> | jq " . | .DataSetSummaries[] | .DataSetId " | while read line; do echo "aws quicksight describe-data-set --aws-account-id <foo-id> --data-set-id ${line} | jq \". |.DataSet | {\"Name\": .Name , \"ConsumedSpiceCapacityInBytes\": .ConsumedSpiceCapacityInBytes}\" "; done > describe-data-set.sh

I then want to run this in python and store the results in a variable. I have imported awscli and have run a subprocess command but it just returns the command and the return status:

import awscli
import subprocess
push=subprocess.stdout(['aws', 'quicksight', 'list-data-sets', '--aws-account-id', '<foo-id>', '--region', '<foo-region>', '|', 'jq', '"', '.', '|', '.DataSetSummaries[]', '|', '.DataSetId', '"', '|', 'while', 'read', 'line;', 'do', 'echo', '"aws', 'quicksight', 'describe-data-set', '--aws-account-id', '<foo-id>', '--data-set-id', '${line}', '|', 'jq', '".', '|.DataSet', '|', '{"Name":', '.Name', ',', '"ConsumedSpiceCapacityInBytes":', '.ConsumedSpiceCapacityInBytes}"', '";', 'done'])

print(push)

Upvotes: 0

Views: 172

Answers (1)

Sam Hartzog
Sam Hartzog

Reputation: 56

As John R pointed out in the comments, boto3 is the way to go if at all possible. It's an AWS supported wrapper for awscli operations in python. Here's the link to the quicksight reference. Some basic tips:

  • You can supply credentials to boto3 via env vars AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN in which case you'd simply import boto3 and instantiate a quicksight client with qs_client = boto3.client('quicksight') OR by using from boto3.session import Session and supplying them directly to the args of a Session() constructor and obtaining a client with [your session instance].client('quicksight').
  • If you go this route, I'd highly recommend installing boto3 type stubs for quicksight via pip install boto3-stubs[quicksight] in your dev environment. Modern IDEs will use that to provide autocomplete hints and save you a ton of time.

For example:

Show autocomplete hints...

If you're wondering what the heck a "ListDataSetsResponseTypeDef" is, you can access that (and all the other TypedDict defs you're sure to encounter) via from mypy_boto3_quicksight.type_defs import ListDataSetsResponseTypeDef.

I realize I haven't answered your question directly, but hopefully this helps. Good luck!

Upvotes: 1

Related Questions