CODEWITHSUNDEEP
sqlamazon-web-servicesamazon-s3amazon-s3-select
rogerwhite
rogerwhite

Reputation: 345

AWS CLI for S3 Select

I have the following code, which is used to run a SQL query on a keyfile, located in a S3 bucket. This runs perfectly. My question is, I do not wish to have the output written over to an output file. Could I see the output on the screen (my preference #1)? If not, what about an ability to append to the output file, rather than over-write it (my preference #2). I am using the AWS-CLI binaries to run this query. If there is another way, I am happy to try (as long as it is within bash)

aws s3api select-object-content \
    --bucket "project2" \
    --key keyfile1 \
    --expression "SELECT * FROM s3object s where Lower(s._1) = '[email protected]'" \
    --expression-type 'SQL' \
    --input-serialization '{"CSV": {"FieldDelimiter": ":"}, "CompressionType": "GZIP"}' \
    --output-serialization '{"CSV": {"FieldDelimiter": ":"}}' "OutputFile"

Upvotes: 9

Views: 8866

Answers (3)

jellycsc
jellycsc

Reputation: 12259

Of course, you can use AWS CLI to do this since stdout is just a special file in linux.

aws s3api select-object-content \
--bucket "project2" \
--key keyfile1 \
--expression "SELECT * FROM s3object s where Lower(s._1) = '[email protected]'" \
--expression-type 'SQL' \
--input-serialization '{"CSV": {"FieldDelimiter": ":"}, "CompressionType": "GZIP"}' \
--output-serialization '{"CSV": {"FieldDelimiter": ":"}}' /dev/stdout

Note the /dev/stdout in the end.

Upvotes: 10

Geoff Johnson
Geoff Johnson

Reputation: 1

I think it opens /dev/stdout twice causing kaos.

Upvotes: -1

John Rotenstein
John Rotenstein

Reputation: 269320

The AWS CLI does not offer such options.

However, you are welcome to instead call it via an AWS SDK of your choice.

For example, in the boto3 Python SDK, there is a select_object_content() function that returns the data as a stream. You can then read, manipulate, print or save it however you wish.

Upvotes: 0

Related Questions