CODEWITHSUNDEEP
pythongoogle-app-enginefile-iopython-2.7blobstore
Kwame
Kwame

Reputation: 1097

how to use blobstore.BlobReader more efficiently

I'm using python 2.7 on GAE, and I'm needing to read files 25 to 35 MB in size. I've uploaded them to Blobstore, and I'm using the following code to read the file:

blob_reader = blobstore.BlobReader(blob_key)
for line in blob_reader:
     # ...

My question is, how can I use BlobReader properties buffer_size and position to do this more efficiently. The docs are not very detailed and I don't have a lot of experience with file I/O. The examples given are: 

# Instantiate a BlobReader for a given Blobstore value, setting the
# buffer size to 1 MB.
blob_reader = blobstore.BlobReader(blob_key, buffer_size=1048576)

and 

# Instantiate a BlobReader for a given Blobstore value, setting the
 # initial read position.
 blob_reader = blobstore.BlobReader(blob_key, position=4194304)

But It's not clear how to use this to make reading the file faster.

Thanks,

Upvotes: 0

Views: 784

Answers (1)

Nick Johnson
Nick Johnson

Reputation: 101149

A larger buffer size will result in fewer blobstore operations when reading lines, which will be more efficient. Simply set the buffer size as large as you comfortably can, and you're sorted.

Upvotes: 1

Related Questions