CODEWITHSUNDEEP
google-app-enginegoogle-cloud-datastore
Jose Florido
Jose Florido

Reputation: 171

Export blob properties from Datastore as binary files

I store pictures as Blob properties in Datastore, I am not using Blobstore, just Blob properties.

Now I want to migrate my app to a different platform and I'd like to get the pictures out as JPEG files. Is there a way of doing this with the bulkloader tool or using the mapper tool that appengine provides?

Thanks!

Update

I tried using the blob_to_file transform helper but could not make it work.

This is my bulkloader.yaml file

python_preamble:
- import: base64
- import: re
- import: google.appengine.ext.bulkload.transform
- import: google.appengine.ext.bulkload.bulkloader_wizard
- import: google.appengine.ext.db
- import: google.appengine.api.datastore
- import: google.appengine.api.users

transformers:

- kind: File
  connector: csv
  connector_options:
      encoding: utf-8
      columns: from_header

  property_map:

      - property: fileId
          external_name: fileId

      - property: bytes_content
          external_name: local_filename
          export_transform: transform.blob_to_file('fileId', 'pictures')

And this is my File model:

class File(db.Model):
    fileId = db.IntegerProperty()
    bytes_content = db.BlobProperty()

I run the bulkloader with the following command:

appcfg.py download_data --application=XXXX --kind=File --url=XXXXX/_ah/remote_api --filename=File.csv --config_file=bulkloader.yaml

At the end of download all I get is a big binary file and the folder pictures is empty. Thanks for your help!

Upvotes: 0

Views: 708

Answers (2)

voscausa
voscausa

Reputation: 11706

You have two other alternatives : 1) Send the data to the other system with URL Fetch 2) If the number of images is not that big, you can download them one by one. I extracted some working code as an example :

    kwis = KindWithImages.get_by_key_name(file_name)              
    if kwis: 
        (content_type, encoding) = mimetypes.guess_type(kwis.file_ext)
        logging.info('download : %s%s content-type %s encoding %s' % (file_name, kwis.file_ext, content_type, encoding))
        if content_type == None :
            self.response.out.write('Content_type unknown for : %s' % (kwis.file_ext))
        else :
            self.response.headers['Content-Type'] = content_type
            self.response.headers['Content-Disposition'] = 'attachment; filename=%s%s' % (file_name, kwis.file_ext)
            self.response.out.write(kwis.content)

Upvotes: 1

schuppe
schuppe

Reputation: 2033

You can use the "Bulkloader Transform Helper function" blob_to_file(). General information can be found at http://code.google.com/appengine/docs/python/tools/uploadingdata.html and Nick Johnson's write-up: http://blog.notdot.net/2010/04/Using-the-new-bulkloader.

For BlobProperty export in particular, check out the bulkloader sample app: http://bulkloadersample.appspot.com/

When you look at the 'bulkloader.yaml' file you will find this:

python_preamble:
- import: google.appengine.ext.bulkload.transform

and this:

# A sample writing a BlobProperty to an external file.
# (Note: not a BlobReferenceProperty.)
- model: models.Attachment
  connector: csv
  connector_options:
    encoding: utf-8
    columns: from_header
  property_map:
    - property: __key__
      external_name: ID
      export_transform: datastore.Key.name
    - property: filename
      external_name: filename
    - property: filecontents
      external_name: local_filename
      export_transform:
        transform.blob_to_file('filename', 'AttachmentBlobs')

Upvotes: 1

Related Questions