CODEWITHSUNDEEP
google-app-engine
Shaun Budhram
Shaun Budhram

Reputation: 3710

Google App Engine - keyword search + ordering on other properties

Say I have an entity that looks a bit like this:

class MyEntity(db.Model):
    keywords            = db.StringListProperty()
    sortProp            = db.FloatProperty()

I have a filter that does a keyword search by doing this:

query = MyEntity.all()\
                .filter('keywords >=', unicode(kWord))\
                .filter('keywords <',  unicode(kWord) + u"\ufffd")\
                .order('keywords')

Which works great. The issue I'm running into is that if I try to put an order on that using 'sortProp':

                .order('sortProp')

ordering has no effect. I realize why - the documentation specifically says this is not possible and that sort order is ignored when using equality filters with a multi-valued property (from the Google docs):

One important caveat is queries with both an equality filter and a sort order on a multi-valued property. In those queries, the sort order is disregarded. For single-valued properties, this is a simple optimization. Every result would have the same value for the property, so the results do not need to be sorted further. However, multi-valued properties may have additional values. Since the sort order is disregarded, the query results may be returned in a different order than if the sort order were applied. (Restoring the dropped sort order would be expensive and require extra indices, and this use case is rare, so the query planner leaves it off.)

My question is: does anyone know of a good workaround for this? Is there a better way to do a keyword search that circumvents this limitation? I'd really like to combine using keywords with ordering for other properties. The only solution I can think of is sorting the list after the query, but if I do that I lose the ability to offset into the query and I may not even get the results with the highest sort order if the data set is large.

Thanks for your tips!

Upvotes: 2

Views: 407

Answers (2)

Ski
Ski

Reputation: 14487

Workaround 1: Apply stemming algorithms for keywords then you won't need to do a comparison look up.

Workaround 2: Store all unique keywords in separate entity group ("table"). From this group find keywords which match your criteria. Then do query with keywords IN [kw1, kw2, ...]. Make sure that the number of matching keywords is not too big, for example you can select only first 10.

Workaround 3: Reorder list of items on application side

Workaround 4: Use IndexTank for full-text search, or apply for "Trusted Tester Program" as mentioned by @proppy.

Upvotes: 2

Nick Johnson
Nick Johnson

Reputation: 101149

Instead of doing prefix matches, properly tokenize, stem and normalize your strings, and do equality comparisons on them.

Upvotes: 1

Related Questions