CODEWITHSUNDEEP
python-3.xamazon-web-servicesamazon-s3aws-lambdaboto3
sam_V
sam_V

Reputation: 49

Difficulty in executing boto3 S3 copy function using aws lambda

Here is the scenario. I have an S3 bucket (e.g. daily-data-input) where daily files will be written to a specific folder (e.g. S3://daily-data-input/data/test/). Whenever a file is written under the "test" folder a copy should also be written to the "test_copy" folder in the same bucket. If "test_copy" is not existing, it should be created.

I have used S3 event notification and attached it to a lambda function(with python 3.7) which will check if the "test_copy" key is existing if not will be created. I am able to create the "test_copy" folder successfully and couldn't make the S3 copy via boto3 to be working.

Here is the code for your reference:

import boto3
import os
import botocore
s3 = boto3.resource('s3')
s3_cli=boto3.client('s3')

def lambda_handler(event, context):

    bucket_name = event ['Records'][0]['s3']['bucket']['name']
    bucket_key = event['Records'][0]['s3']['object']['key']
    file = (os.path.basename(bucket_key))
    source_key_path = (os.path.dirname(bucket_key))
    target_keypath = source_key_path+'_'+'copy'+'/'
    target_bucket_key = target_keypath+file
    copy_source = {'Bucket': bucket_name, 'Key': bucket_key}
    try:
        s3.Object(bucket_name, target_keypath).load()
    except botocore.exceptions.ClientError as e:
        if e.response['Error']['Code'] == "404":
            # Create the key
            print ("Creating target _copy folder")
            s3_cli.put_object(Bucket=bucket_name,Key=target_keypath)
            #copy the file
            #s3.copy_object(Bucket=bucket_name, Key=target_bucket_key, CopySource=copy_source)
        else:
            print ("Something went wrong!!")
    else:
        print ("Key exists!!")
        # s3.copy_object(Bucket=bucket_name, Key=target_bucket_key, CopySource=copy_source)

I tried s3.copy_object, s3_cli.meta.client.copy, bucket.copy() and none of them are working. Please let me know if i am doing something wrong.

Upvotes: 1

Views: 1469

Answers (1)

jarmod
jarmod

Reputation: 78563

Here is one simple way to copy an object in S3 within a bucket:

import boto3
s3 = boto3.resource('s3')

bucket = 'mybucket'
src_key = 'data/test/cat.png'
dest_key = 'data/test_copy/cat.png'

s3.Object(bucket, dest_key).copy_from(CopySource=f'{bucket}/{src_key}')

Here is another, lower-level way to do the same thing:

import boto3
s3 = boto3.client('s3')

bucket = 'mybucket'
src_key = 'data/test/cat.png'
dest_key = 'data/test_copy/cat.png'

s3client.copy_object(Bucket=bucket, CopySource={'Bucket':bucket,'Key':src_key}, Key=dest_key)

Upvotes: 2

Related Questions