CODEWITHSUNDEEP
pythonamazon-web-servicesamazon-s3boto3boto
Abe Mused
Abe Mused

Reputation: 65

Searching s3 for a bucket using boto3

I'm trying to create a python script that uploads a file to an s3 bucket. The catch is that I want this script to go to s3 and search through all the buckets and find a bucket that contains a certain keyword in its name and upload the file to that bucket.

I currently have this:

import boto3
import json

BUCKET_NAME = 'myBucket'

with open('my-file.json', 'rb') as json_file:
    data = json.load(json_file)
    
    s3 = boto3.resource('s3')
    s3.Bucket(BUCKET_NAME).put_object(Key='banner-message.json', Body=json.dumps(data))
    print ("File successfully uploaded.")

This script successfully uploads the file to s3. But, as you can see, the bucket name I pass in has to be the same exact match as the s3 bucket. I want to be able to search through all the buckets in s3 and find the bucket that contains the keyword I pass in.

For example, in this case, I want to be able to pass in 'myBucke' and have it search s3 for a bucket that contain that. 'myBucket' contains 'myBucke', so it uploads it to that. Is this possible?

Upvotes: 3

Views: 3079

Answers (2)

Abe Mused
Abe Mused

Reputation: 65

The previous anaswer works but I ended up using this:

def findBucket(s3):
    for bucket in s3.buckets.all():
            if('myKeyWord' in bucket.name):
                return bucket.name
    return 'notFound'

s3 = boto3.resource('s3')
bucketName = findBucket(s3)
if(bucketName != 'notFound'):
    #upload file to that bucket

Upvotes: 1

Shawn
Shawn

Reputation: 9472

You can call the list_buckets API. It "Returns a list of all buckets owned by the authenticated sender of the request." https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/s3.html#S3.Client.list_buckets

Once you have the list, you can loop through it to check each bucket name to see if it matches the keyword. Perhaps something like this:

s3_client = boto3.client('s3')
buckets = s3_client.list_buckets()['Buckets']
for bucket in buckets:
    bucket_name = bucket['Name']
    if 'keyword' in bucket_name:
        # do your logic to upload

Upvotes: 2

Related Questions