CODEWITHSUNDEEP
google-cloud-storagebucket
Mpizos Dimitris
Mpizos Dimitris

Reputation: 5001

moving local data to google cloud bucket using python api

I can move data in google storage to buckets using the following:

gsutil cp afile.txt gs://my-bucket

How to do the same using the python api library:

from google.cloud import storage
storage_client = storage.Client()

# Make an authenticated API request
buckets = list(storage_client.list_buckets())
print(buckets)

Cant find anything more than the above.

Upvotes: 1

Views: 2258

Answers (1)

Paul
Paul

Reputation: 558

There is an API Client Library code sample code here. My code typically looks like below which is a slight variant on the code they provide:

from google.cloud import storage

client = storage.Client(project='<myprojectname>')
mybucket = storage.bucket.Bucket(client=client, name='mybucket')
mydatapath = 'C:\whatever\something' + '\\'   #etc
blob = mybucket.blob('afile.txt')
blob.upload_from_filename(mydatapath + 'afile.txt')

In case it is of interest, another method is to run the "gsutil" command line how you have typed in your Original Post using the subprocess command, e.g.:

import subprocess
subprocess.call("gsutil cp afile.txt gs://mybucket/", shell=True)

In my view, there are pros and cons of both methods depending on what you are trying to achieve - the latter method allows multi-threading if you have many files to upload whereas the former method perhaps allows better control, specification of metadata for each file, etc.

Upvotes: 4

Related Questions