CODEWITHSUNDEEP
google-app-enginegoogle-cloud-datastore
somebody
somebody

Reputation:

How does one get a count of rows in a Datastore model in Google App Engine?

I need to get a count of records for a particular model on App Engine. How does one do it?

I bulk uploaded more than 4000 records but modelname.count() only shows me 1000.

Upvotes: 28

Views: 28285

Answers (8)

M. Schena
M. Schena

Reputation: 2107

Another solution is using a key only query and get the size of the iterator. The computing time with this solution will rise linearly with the amount of entrys:

Datastore datastore = DatastoreOptions.getDefaultInstance().getService();
KeyFactorykeyFactory = datastore.newKeyFactory().setKind("MyKind");
Query query = Query.newKeyQueryBuilder().setKind("MyKind").build();
int count = Iterators.size(datastore.run(query));

Upvotes: 2

svpino
svpino

Reputation: 1994

This is a very old thread, but just in case it helps other people looking at it, there are 3 ways to accomplish this:

  1. Accessing the Datastore statistics
  2. Keeping a counter in the datastore
  3. Sharding counters

Each one of these methods is explained in this link.

Upvotes: 9

varun
varun

Reputation: 4650

DatastoreService ds = DatastoreServiceFactory.getDatastoreService();
Query query = new Query("__Stat_Kind__");
Query.Filter eqf = new Query.FilterPredicate("kind_name",
                                Query.FilterOperator.EQUAL,
                                "SomeEntity");
query.setFilter(eqf);
Entity entityStat = ds.prepare(query).asSingleEntity();
Long totalEntities = (Long) entityStat.getProperty("count");

Upvotes: 2

TommyN
TommyN

Reputation: 2381

You should use Datastore Statistics:

Query query = new Query("__Stat_Kind__");
query.addFilter("kind_name", FilterOperator.EQUAL, kind);       
Entity entityStat = datastore.prepare(query).asSingleEntity();
Long totalEntities = (Long) entityStat.getProperty("count");

Please note that the above does not work on the development Datastore but it works in production (when published).

I see that this is an old post, but I'm adding an answer in benefit of others searching for the same thing.

Upvotes: 22

tkc
tkc

Reputation: 101

count = modelname.all(keys_only=True).count(some_upper_limit)

Just to add on to the earlier post by dar, this 'some_upper_limit' has to be specified. If not, the default count will still be a maximum of 1000.

Upvotes: 8

Gavin
Gavin

Reputation: 5871

I still hit the 1000 limit with count so adapted dar's code (mine's a bit quick and dirty):

class GetCount(webapp.RequestHandler):
    def get(self):
        query = modelname.all(keys_only=True)

        i = 0
        while True:
            result = query.fetch(1000)
            i = i + len(result)
            if len(result) < 1000:
                break
            cursor = query.cursor()
            query.with_cursor(cursor)

        self.response.out.write('<p>Count: '+str(i)+'</p>')

Upvotes: 3

dar
dar

Reputation: 6530

As of release 1.3.6, there is no longer a cap of 1,000 on count queries. Thus you can do the following to get a count beyond 1,000:

count = modelname.all(keys_only=True).count()

This will count all of your entities, which could be rather slow if you have a large number of entities. As a result, you should consider calling count() with some limit specified:

count = modelname.all(keys_only=True).count(some_upper_bound_suitable_for_you)

Upvotes: 13

Federico Builes
Federico Builes

Reputation: 5097

In GAE a count will always make you page through the results when you have more than 1000 objects. The easiest way to deal with this problem is to add a counter property to your model or to a different counters table and update it every time you create a new object.

Upvotes: 6

Related Questions