CODEWITHSUNDEEP
amazon-web-servicesamazon-s3boto3
Lijo Jose
Lijo Jose

Reputation: 337

Specify wild character in S3 filter prefix

I have files in S3 with specified folder structure like : Year/Month/Date/file.csv .

I use below code to fetch data for particular date.

import boto3
resource = boto3.resource('s3')
root_data = resource.Bucket('my_bucket')
for obj in root_data.objects.filter(Prefix='2018/09/19'):
    process(obj)

I want to know is it possible to fetch data by specifying wild characters in prefix (or any other way)??

eg: To get list of files for a particular day of all month like Prefix='2018/*/19'. I have tried with '*' in above code, but is not fetching any file names.

Note: I cant change folder structure of S3

Upvotes: 1

Views: 2672

Answers (1)

Rez Moss
Rez Moss

Reputation: 4604

You have two options, First:

The prefix and delimiter parameters limit the kind of results returned by a list operation. Prefix limits results to only those keys that begin with the specified prefix and delimiter causes the list to roll up all keys that share a common prefix into a single summary list result.

For more info read the following page: https://docs.aws.amazon.com/AmazonS3/latest/dev/ListingKeysHierarchy.html

Second: Get all objects and check them on Python

import boto3
s3 = boto3.resource('s3')
bucket = s3.Bucket('bucket')
for obj in bucket.objects.all():
    if '.pdf' in obj.key:
        print(obj.key)

Upvotes: 1

Related Questions