CODEWITHSUNDEEP
databasegoogle-app-engineindexing
Nyambaa
Nyambaa

Reputation: 41887

About indexes of GAE datastore

I have a following model in the GAE app.

class User
    school_name = db.StringProperty(Indexed=True)
    country = db.StringProperty(Indexed=True)
    city = db.StringProperty(Indexed=True)
    sex = db.StringProperty(Indexed=True)
    profession = db.StringProperty(Indexed=True)
    joined_date = db.DateTimeProperty(Indexed=True)

And I want to filter the users by combinations of these fields. Result of the filter should show a user at first who is joined recently. So which means any query end by order operation, I suppose. like that: 

User.all().filter('country =','US').filter('profession =','SE').order('-joined_date')
User.all().filter('school_name =','AAA').filter('profession =','SE').order('-joined_date')
....
User.all().filter('sex =','Female').filter('profession =','HR').order('-joined_date')

All these fields combination would be C(5,1)+C(5,2)+...+C(5,5) = 31.

My question is to implement it, do I need to create indexes for all these cases(31) in the Google AppEngine. Or can you suggest other way to implement it?

Note: C(n,k) is combination formula, see more on http://en.wikipedia.org/wiki/Combination

Thanks in advance!

Upvotes: 2

Views: 212

Answers (1)

Nick Johnson
Nick Johnson

Reputation: 101149

You have several options:

  1. Create all 31 indexes, as you suggest.
  2. Do the sorting in memory. Without a sort order, all your queries can be executed with the built-in merge-join strategy, and so you won't need any indexes at all.
  3. Restrict queries to those that are more likely, or those that eliminate most of the non-matching results, and perform additional filtering in memory.
  4. Put all your data in a ListProperty for indexing as "key:value" strings, and filter only on that. You will need to create multiple indexes with different occurrence counts on that field (eg, indexing it once, twice, etc), and it will result in the same number of index entries, but fewer custom indexes used.

Upvotes: 4

Related Questions