CODEWITHSUNDEEP
pythonpython-3.xgoogle-cloud-platform
alcor
alcor

Reputation: 565

Retrieve buckets with python on GCP virtual machine

I'm trying to use Python3 on a GCP virtual machine (not in local machine). I need to extract data from a certain bucket. This is the code i'm using:

from google.cloud import storage
client = storage.Client()
bucket_name = timetables-new/23/00_H0.json
bucket = client.get_bucket("bucket_name")

It returns the following error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/username/.local/lib/python3.6/site-packages/google/cloud/storage/client
.py", line 301, in get_bucket
    bucket.reload(client=self)
  File "/home/username/.local/lib/python3.6/site-packages/google/cloud/storage/_helpe
rs.py", line 130, in reload
    _target_object=self,
  File "/home/username/.local/lib/python3.6/site-packages/google/cloud/_http.py", lin
e 393, in api_request
    raise exceptions.from_http_response(response)
google.api_core.exceptions.NotFound: 404 GET https://www.googleapis.com/storage/v1/b/timetables-new/23/00_H0.json?projection=noAcl: Not Found

How can i retrieve the file, so i can use it in my python script?

Edit: Since i was using an object name as bucket name (/ not allowed) i also tried: 

bucket_name = timetables-new

Which is the name of the entire bucket, but i have the same error.

Upvotes: 1

Views: 533

Answers (2)

Aladejubelo Oluwashina
Aladejubelo Oluwashina

Reputation: 428

One thing that is missing from you post is how you are actually authenticating. You may need to set the json key of service account as follows:

GOOGLE_APPLICATION_CREDENTIALS=key.json

Upvotes: 1

KRYSEF
KRYSEF

Reputation: 98

try to use Gcsfuse of Gsutil, it goes something like this :

  • gsutil lets you copy files from the bucket to your virtual machine : 
gsutil -m cp gs://name_of_bucket/.../file destination/path/on/your/VM
  • Whereas gcsfuse creates a symbolic link to allow to access files in the bucket :
mkdir /path/to/mount
gcsfuse bucket_name /path/to/mount

Upvotes: 0

Related Questions