CODEWITHSUNDEEP
google-app-enginegoogle-cloud-datastore
csytan
csytan

Reputation: 198

Query response size limit on appengine?

Appengine docs mention a 1Mb limit on both entity size and batch get requests (db.get()): http://code.google.com/appengine/docs/python/datastore/overview.html

Is there also a limit on the total size of all entities returned by a query for a single fetch() call?

Example query:

db.Model.all().fetch(1000)

Update: As of 1.4.0 batch get limits have been removed!

Upvotes: 5

Views: 2764

Answers (3)

csytan
csytan

Reputation: 198

I tried it out on production and you can indeed exceed 1 Mb total for a query. I stopped testing at around 20 Mb total response size.

from app import models

# generate 1Mb string
a = 'a'
while len(a) < 1000000:
    a += 'a'

# text is a db.TextProperty()
c = models.Comment(text=a)
c.put()


for c in models.Comment.all().fetch(100):
    print c

Output:

<app.models.Comment object at 0xa98f8a68a482e9f8>
<app.models.Comment object at 0xa98f8a68a482e9b8>
<app.models.Comment object at 0xa98f8a68a482ea78>
<app.models.Comment object at 0xa98f8a68a482ea38>
....

Upvotes: 3

Robert Kluin
Robert Kluin

Reputation: 8292

Yes there is a size limit; the quotas and limits section explicitly states there is a 1 megabyte limit to db API calls.

You will not be able to db.get(list_of_keys) if the total size of the entities in the batch is over 1 megabyte. Likewise, you will not be able to put a batch if the total size of the entities in the batch is over 1 megabyte.

The 1,000 entity limit has been removed, but (at present) you will need to ensure the total size of your batches is less than 1 megabyte yourself.

Upvotes: 2

Karl Rosaen
Karl Rosaen

Reputation: 4644

Theres no longer a limit on the number of entities that can be returned by a query, but the same entity size limit applies when you are actually retrieving / iterating over the entities. This will only be on a single entity at a time though; it is not a limit on the total size of all entities returned by the query.

Bottom line: as long as you don't have a single entity that is > 1Mb you should be OK with queries.

Upvotes: 6

Related Questions