CODEWITHSUNDEEP
amazon-s3boto3ibm-cloud-storage
mark86v1
mark86v1

Reputation: 312

PDF files not getting copied in IBM COS properly

I am using Python SDK to copy files from one IBM COS bucket to another.

Source bucket format:

folder/subfolder1/subfolder2/.json/pdf/html

The above is the format of the source bucket. While copying the objects to the destination bucket also I am keeping the same folder structure.

The copy happens in a loop since there are more number of files to process. The problem is, sometimes, most of the .pdf files are not copied properly (say 100KB file is copied as 8KB) but 150KB file is copied correctly. What could be the problem here?

I have tried to put sleep time of 5s between consecutive copies, that too did not work. I have tried direct copy with cos.copy_object as well as with get_object and upload_fileobj like below. Both seems not to work.

def copy_file_between_buckets(source_bucket, destination_bucket, file_key, dest_key=None):
    try:
        if dest_key is None:
            dest_key = file_key  
        file_obj = cos.get_object(Bucket=source_bucket, Key=file_key)
        file_stream = io.BytesIO(file_obj['Body'].read())  # Stream the file content in memory

        cos.upload_fileobj(file_stream, destination_bucket, dest_key)
        print(f"Copied {file_key} to {destination_bucket} as {dest_key}")
    except ClientError as e:
        print(f"Unable to copy {file_key}: {e}")

PS: Tagging s3 and boto3 as IBM COS Python SDK is built on top of boto3.

I tried to copy .pdf files between IBM COS buckets but the copy did not happen properly.

Upvotes: 0

Views: 35

Answers (0)

Related Questions