CODEWITHSUNDEEP
python-3.xamazon-web-servicesamazon-s3boto3
user3644850
user3644850

Reputation: 3

S3 Python List nested sub directories

How can I list sub directories within a Prefix of a S3 bucket using boto3 python?

For example, if I have bucket named test with following structure:

test/abc/def/1/a.gz
test/abc/def/2/b.gz
test/abc/ghi/1/a.gz

Then I want the output as:

test/abc/def/1
test/abc/def/2
test/abc/ghi/1

Upvotes: 0

Views: 995

Answers (1)

John Rotenstein
John Rotenstein

Reputation: 270194

Folders/subdirectories don't actually exist in S3. Instead, they form part of the filename (Key) of an object.

Therefore, just grab the Key up to the last slash:

import boto3

s3 = boto3.client('s3', region_name = 'ap-southeast-2')

response = s3.list_objects_v2(Bucket='my-bucket')

# If the Key contains a slash, store the Key up to the last slash
folders = set(object['Key'][:object['Key'].rfind('/')+1] for object in response['Contents'] if '/' in object['Key'])

# Print the results
print('\n'.join(sorted(folders)))

See also: Determine if folder or file key - Boto

Upvotes: 1

Related Questions