CODEWITHSUNDEEP
pythonamazon-web-servicesamazon-s3boto3
KRico
KRico

Reputation: 641

Using boto3 to filter s3 objects so that caller is not filtering

My s3 bucket has a thousands of files. They are all in the same "folder". So the s3 prefix is the same. I want to use python boto3 to get a list of filenames that contains a certain word. I dont want the boto3 call to send back all the filenames and have the client filter out the names. I seen example from using "yield" and ".filter" but those are receiving all the files and making the client do a lot of work.

To help give a better understanding, if I use the AWS CLI:

aws --profile test s3api list-objects-v2 --bucket mybucket --prefix tmp/ --output json --query "Contents[?contains(Key, 'foobar')]"

BUT I need to send a request using boto3 and AWS just send the filenames back with "foobar" in them.

Upvotes: 2

Views: 824

Answers (1)

Marcin
Marcin

Reputation: 238101

BUT I need to send a request using boto3 and AWS just send the filenames back with "foobar" in them

You can't do this with regular boto3 s3 API calls as this is not how that API works. So if you don't want to get all the names first, and then filter them out by yourself, then there is no way to achieve what you want with just a single boto3 request.

The only help would maybe be from Amazon S3 inventory. So you could request the inventory, get the resulting CSV file, and filter that. But still you would have to filter it yourself.

Upvotes: 5

Related Questions