CODEWITHSUNDEEP
python-3.xamazon-s3boto3
RustyShackleford
RustyShackleford

Reputation: 3667

How to count csv files with specific naming structure in s3 bucket using boto3?

I have files with many different names in a s3 bucket.

I want to count how many csv's with the word 'member' are in my bucket?

However the member files have an UUID attached them like so:

member_asldf2323209.csv

I have tried this so far:

import boto3

# create the s3 resource
s3 = boto3.resource('s3')

# get the file object
obj = s3.Object('bucket_name', 'key')

# read the file contents in memory
file_contents = obj.get()["Body"].read()

# print the occurrences of the new line character to get the number of lines
print file_contents.count('\n')

this only gets me one 'member' file with no UUID attached.

Upvotes: 1

Views: 640

Answers (1)

John Rotenstein
John Rotenstein

Reputation: 269091

If you wish to count the number of objects that contain a particular word in the Key, you could use something like:

import boto3

s3_client = boto3.client('s3', region_name = 'ap-southeast-2')

listing = s3_client.list_objects_v2(Bucket='my-bucket')

members = [object['Key'] for object in listing['Contents'] if 'member' in object['Key']]
print (members)
print (len(members))

Upvotes: 2

Related Questions