CODEWITHSUNDEEP
pythonamazon-s3boto3
Srinivas
Srinivas

Reputation: 2100

Cannot upload s3 files to another region (clients bucket) despite successful response

This is my code. I am trying to copy a directory from one bucket to another. I am seeing everything is positive, but files are not appearing in the clients bucket.

import boto3

ACCESS_KEY = 'access_key'
SECRET_KEY = 'secret_key'
REGION_NAME = 'US_EAST_1'

source_bucket = 'source_bucket'

#Make sure you provide / in the end 
source_prefix = 'source_prefix'

target_bucket = 'target-bucket'
target_prefix = 'target-prefix'

client = boto3.client('s3')
session_src = boto3.session.Session()
source_s3_r = session_src.resource('s3')

def get_s3_keys(bucket, prefix):
    keys = []
    response = client.list_objects_v2(Bucket=bucket,Prefix=prefix,MaxKeys=100)
    for obj in response['Contents']:
        keys.append(obj['Key'])
    return keys


session_dest = boto3.session.Session(aws_access_key_id=ACCESS_KEY, 
aws_secret_access_key=SECRET_KEY)
dest_s3_r = session_dest.resource('s3')

# create a reference to source image
old_obj = source_s3_r.Object(source_bucket, source_prefix)

# create a reference for destination image
new_obj = dest_s3_r.Object(target_bucket, target_prefix)

keys = get_s3_keys(source_bucket, source_prefix)

responses = []
# upload the image to destination S3 object
for filename in keys:
    print("Transferring file {}, {}".format(source_bucket,filename))
    old_obj = source_s3_r.Object(source_bucket, filename)
    response = new_obj.put(Body=old_obj.get()['Body'].read())
    response_code = response['ResponseMetadata']['HTTPStatusCode']
    responses.append(response_code)
    print("File transfer response {}".format(response_code))

distinct_response = list(set(responses))

if len(distinct_response) > 1 or distinct_response[0] != 200:
    print("File could not be transfered to krux bucket. Exiting now")
    exit(1)
else:
    print("File transfer to krux bucket successful")

I am getting a successful response code of 200 but the file is not transferred across.

Upvotes: 0

Views: 53

Answers (1)

r0ck
r0ck

Reputation: 192

Srinivas, Try this I used S3 Resource object, try equivalent S3 Client if you want...

bucket= s3.Bucket(bucket_name)    #from_bucket
for osi in bucket.objects.all():    
  print(osi)
  copy_source={
    'Bucket': bucket.name, 
    'Key': osi.key 
    }   
  s3.Bucket('to_bucket').copy(copy_source, osi.key)

Hope it helps.. r0ck

Upvotes: 1

Related Questions