CODEWITHSUNDEEP
google-cloud-platformgoogle-cloud-storagegsutil
Vicky
Vicky

Reputation: 1

GCP Storage - How to create a script that copies all files/folders from one bucket to another, but in a different folder structure

There is a bucket on GCP in which the folders are named as below.

Tests_111/tests.pdf

Tests_112/tests.pdf

AllTests_111/alltests.pdf
...

These folders need to be copied into another bucket in the following structure.

Tests/111/tests.pdf

Tests/112/tests.pdf

AllTests/111/alltests.pdf

I basically, just trying to check the folder name to see if it can be spliced by '_', which will give me "Tests" & "111". After that, create a folder with the name "Tests" and a subfolder with the name "111" within that one.

I'm new to GCP and gsutils. Is there any way to achieve this?

Upvotes: 0

Views: 373

Answers (1)

MBHA Phoenix
MBHA Phoenix

Reputation: 2217

here an example of code snippet to copy objects from one bucket to another

from google.cloud import storage


def copy_blob(
    bucket_name, blob_name, destination_bucket_name, destination_blob_name
):
    """Copies a blob from one bucket to another with a new name."""
    # bucket_name = "your-bucket-name"
    # blob_name = "your-object-name"
    # destination_bucket_name = "destination-bucket-name"
    # destination_blob_name = "destination-object-name"

    storage_client = storage.Client()

    source_bucket = storage_client.bucket(bucket_name)
    source_blob = source_bucket.blob(blob_name)
    destination_bucket = storage_client.bucket(destination_bucket_name)

    blob_copy = source_bucket.copy_blob(
        source_blob, destination_bucket, destination_blob_name
    )

    print(
        "Blob {} in bucket {} copied to blob {} in bucket {}.".format(
            source_blob.name,
            source_bucket.name,
            blob_copy.name,
            destination_bucket.name,
        )
    )

Ref: https://cloud.google.com/storage/docs/copying-renaming-moving-objects#storage-copy-object-python

So you have only to code the logic to set properly the destination_blob_name from the blob_name. By the way a blob name is fully qualified, means it include the folder name and the file name like Tests/111/tests.pdf

Upvotes: 1

Related Questions