CODEWITHSUNDEEP
google-app-engineblobstoregae-eclipse-plugin
Katedral Pillon
Katedral Pillon

Reputation: 14844

Fetching blob from Google Blobstore without knowing size before hand

I need to fetch a blob from the Blobstore programmatically, without knowing the size before hand. Does anyone know how to do that?

I have tried using 

BlobstoreService blobStoreService = BlobstoreServiceFactory.getBlobstoreService();
byte[] picture = blobStoreService.fetchData(blobKey, 0, Integer.MAX_VALUE);

but I get an error since (at least seemingly) Integer.MAX_VALUE is too big.

java.lang.IllegalArgumentException: Blob fetch size 2147483648 it larger than maximum size 1015808 bytes.
at com.google.appengine.api.blobstore.BlobstoreServiceImpl.fetchData(BlobstoreServiceImpl.java:250)

So does anyone know how to do this correctly? Also if you could tell me in passing, is it better to same images into the blobstore as "jpeg" or as "png"?

Upvotes: 2

Views: 1382

Answers (3)

Kamel
Kamel

Reputation: 1866

def blob_fetch(blob_key):
  blob_info = blobstore.get(blob_key)
  total_size = blob_info.size
  unit_size = blobstore.MAX_BLOB_FETCH_SIZE
  pos = 0
  buf = cStringIO.StringIO()
  try:
    while pos < total_size:
      buf.write(blobstore.fetch_data(blob_key, pos, min(pos + unit_size - 1, total_size)))
      pos += unit_size
    return buf.getvalue()
  finally:
    buf.close()

The MAX_BLOB_FETCH_SIZE is not obvious in the document.

Upvotes: 2

S&#233;bastien Trottier
S&#233;bastien Trottier

Reputation: 2286

In Python:

from google.appengine.ext.blobstore import BlobInfo
from google.appengine.api import blobstore
import cStringIO as StringIO

blobinfo = BlobInfo.get(KEY)

offset = 0
accumulated_content = StringIO.StringIO()
while True:
  fetched_content = blobstore.fetch_data(
      blobinfo.key(),
      offset,
      offset + blobstore.MAX_BLOB_FETCH_SIZE - 1)
  accumulated_content.write(fetched_content)
  if len(fetched_content) < blobstore.MAX_BLOB_FETCH_SIZE:
    break
  offset += blobstore.MAX_BLOB_FETCH_SIZE

Upvotes: 1

Pablo Chvx
Pablo Chvx

Reputation: 1931

Hope this helps, this is the way I have been doing it for a while:

        BlobstoreService blobstoreService = BlobstoreServiceFactory.getBlobstoreService();
        BlobKey blobKey = new BlobKey(KEY);

        // Start reading
        ByteArrayOutputStream out = new ByteArrayOutputStream();

        long inxStart = 0;
        long inxEnd = 1024;
        boolean flag = false;

        do {
            try {
                byte[] b = blobstoreService.fetchData(blobKey,inxStart,inxEnd);
                out.write(b);

                if (b.length < 1024)
                    flag = true;

                inxStart = inxEnd + 1;
                inxEnd += 1025;

            } catch (Exception e) {
                flag = true;
            }

        } while (!flag);

        byte[] filebytes = out.toByteArray();

I used to use:

BlobInfo blobInfo = blobInfoFactory.loadBlobInfo(blobKey);
filesize = blobInfo.getSize();

to obtaint the size, but for some reason, sometimes this info was null.

Maybe all this can give you an idea.

Upvotes: 3

Related Questions