CODEWITHSUNDEEP
jsonamazon-web-servicescommand-line-interfaceaws-cli
FirstName
FirstName

Reputation: 417

AWS CLI Command

I'm trying to execute the following command using AWS CLI command - 

aws s3 cp s3://my_bucket/folder/file_1234.txt -| pipe to sed command | pipe to jq command | aws s3 cp - s3://my_bucket/new_folder/final_file.txt

The above code is working fine - basically pulling data from s3, doing some operations and pushing it back to s3.

Now, I have some files in s3 that have a pattern - for instance - file_771.txt, file_772.txt, file_773.txt and so on.

Now in order to get all the files that match the pattern I'm doing the following operation which is not working as expected. Its generating an empty output file in s3.

aws s3 cp --include file_77* s3://my_bucket/folder/ -| pipe to sed command | pipe to jq command | aws s3 cp - s3://my_bucket/new_folder/final_file.txt

This code is generating empty final_file.txt. Any reason ? Am I missing something in the code ?

Upvotes: 0

Views: 281

Answers (1)

Dusan Bajic
Dusan Bajic

Reputation: 10869

To copy multiple files at once, you would have to use --recursive, in your case with --exclude "*" --include "file_77*", but:

Downloading as a stream is not currently compatible with the --recursive parameter

cp

Upvotes: 1

Related Questions