CODEWITHSUNDEEP
pythongoogle-app-enginegoogle-cloud-storage
Eric G
Eric G

Reputation: 1292

Google App Engine - using Google Cloud Storage in separate project, project has special characters in name

I am trying to read and write from a GAE (Python) app to GCS bucket in a separate project. The other project:

However, I am not able to write to the bucket from my GAE app. The project (which I did not set up) has a colon (:) in the Project ID, and I get the error ValueError: Path should have format /bucket/filename....

I tried using the url-escaped version of the Project ID (i.e., in the first segment of the bucket path), and got the same error.

I also tried using the Project Number as the first segment of the path, but then it could not find the bucket: NotFoundError: Expect status [201] from Google Storage. But got status 404..

Is it possible to reference such a bucket? Do I need to use the x-goog-project-id header somehow?

EDIT: GCS adapter code

Note that the bucket is set reading from a config file, it ends up being something like Uploader('my:gcs-project/folder'). I am sure it is set correctly because of the error message I get.

import os.path
import cloudstorage as gcs

class Uploader():

  def __init__(self,bucket):
    self._bucket = bucket
    self._path   = ''
    self._meta   = {}
    self._type   = None

  def datasend(self, id):
    self._path = id
    return self

  def description(self, _):
    self._meta['x-goog-meta-desc'] = _
    return self

  def type(self, _):
    self._type = _
    return self

  def fullpath(self, filename):
    return "/".join([
             part for part in [
               "", self._bucket, self._path, os.path.basename(filename)
             ] if not (part is None)
           ])

  def __call__(self, file):
    self.upload(file)

  def upload(self, file):
    gcs_file = None
    try:
      gcs_file = gcs.open( self.fullpath(file.filename),
                           'w',
                           content_type=self._type,
                           options=self._meta
                         )
      for chunk in fbuffer(file.file):
        gcs_file.write(chunk)

    finally:
      if gcs_file:
        gcs_file.close()

def fbuffer(f, size=10240):
  while True:
    chunk = f.read(size)
    if not chunk: break
    yield chunk

Upvotes: 1

Views: 1288

Answers (1)

Celio Marcos
Celio Marcos

Reputation: 151

Try get your bucket with something like this:

bucket_name = os.environ.get('BUCKET_NAME', app_identity.get_default_gcs_bucket_name())
bucket = '/' + bucket_name
filename = bucket + '/' + "myfile.jpg"

Upvotes: 1

Related Questions