CODEWITHSUNDEEP
google-cloud-platformgoogle-cloud-storagegcp-ai-platform-notebook
naveen kumar
naveen kumar

Reputation: 61

How do I change the extension of multiple files in Cloud Storage?

How do I change the extension of multiple files in GCP's Cloud Storage?

For example, in the following directory:

gs://[bucket-name]/filesdirectory/

I want to change the files with extension .ipynb to .py.

Upvotes: 3

Views: 2866

Answers (2)

Pievis
Pievis

Reputation: 1994

You can do it by using gsutil which you can install on your local machine or use directly from cloud shell.

gsutil uses command much similar to the linux CLI, you can use the gsutil mv command to achieve this, but since you can't use wildcards there you have to use something similar to this:

IFS=$'\n'
gsutil ls gs://your-bucket/*.ipynb| while read x; do gsutil mv $x $(echo $x | sed "s/.ipynb/.py/g"); done

I'm not a shell expert so probably this can be improved, but here's an explanation:

  • gsutil ls uses a wildcard to return the files you want to rename
  • loop through the file and store into a variable the result
  • use gsutil mv + sed to place the file extension and rewrite the file as desired

This is like "rewriting" the files entirely since gcs objects are immutable, so there are probably a few considerations that you should keep in mind, although this might not be your case:

  • if you have ACLs rules specified for those files, you have to use the -p flag to pass them on to the new files
  • these are operations for GCS, implying costs based on your storage class. (since mv is actually copy + delete, if you are on nearline or coldline, you could have additional early deletion fees)

hope this helps :)

Upvotes: 6

ericcco
ericcco

Reputation: 776

As mentioned in the documentation, you can rename your files in your GCS buckets using Console, GSutil command, Code or REST APIS.

The Gsutil command you should use is the following:

gsutil mv gs://[BUCKET_NAME]/[OLD_OBJECT_NAME] gs://[BUCKET_NAME]/[NEW_OBJECT_NAME]

Furthermore, in case you want to change more than one file, I would suggest you to use a script in order to do it for each file you need to change.

Upvotes: 0

Related Questions