CODEWITHSUNDEEP
google-app-enginegoogle-cloud-datastorego
Alexander Voloshyn
Alexander Voloshyn

Reputation: 924

Search entries in Go GAE datastore using partial string as a filter

I have a set of entries in the datastore and I would like to search/retrieve them as user types query. If I have full string it's easy:

q := datastore.NewQuery("Products").Filter("Name =", name).Limit(20)

but I have no idea how to do it with partial string, please help.

Upvotes: 2

Views: 1621

Answers (3)

proppy
proppy

Reputation: 10504

If the number of keywords is limited you could consider adding an indexed list property of partial search strings.

Note that you are limited to 5000 indexes per entity, and 1MB for the total entity size.

But you could also wait for Cloud SQL and Full Text Search API to be avaiable for the Go runtime.

Upvotes: 0

asdf_enel_hak
asdf_enel_hak

Reputation: 7630

q := datastore.NewQuery("Products").Filter("Name >", name).Limit(20)

There is no like operation on app engine but instead you can use '<' and '>'

example:

'moguz' > 'moguzalp'

Upvotes: 4

Adam Crossland
Adam Crossland

Reputation: 14213

EDIT: GAH! I just realized that your question is Go-specific. My code below is for Python. Apologies. I'm also familiar with the Go runtime, and I can work on translating to Python to Go later on. However, if the principles described are enough to get you moving in the right direction, let me know and I wont' bother.

Such an operation is not directly supported on the AppEngine datastore, so you'll have to roll your own functionality to meet this need. Here's a quick, off-the-top-of-my-head possible solution:

class StringIndex(db.Model):
    matches = db.StringListProperty()

    @classmathod
    def GetMatchesFor(cls, query):
        found_index = cls.get_by_key_name(query[:3])
        if found_index is not None:
            if query in found_index.matches:
                # Since we only query on the first the characters,
                # we have to roll through the result set to find all
                # of the strings that matach query. We keep the
                # list sorted, so this is not hard.
                all_matches = []
                looking_at = found_index.matches.index(query)
                matches_len = len(foundIndex.matches)

                while start_at < matches_len and found_index.matches[looking_at].startswith(query):
                    all_matches.append(found_index.matches[looking_at])
                    looking_at += 1

                return all_matches

        return None

    @classmethod
    def AddMatch(cls, match) {
        # We index off of the first 3 characters only
        index_key = match[:3]

        index = cls.get_or_insert(index_key, list(match))
        if match not in index.matches:
            # The index entity was not newly created, so
            # we will have to add the match and save the entity.
            index.matches.append(match).sort()
            index.put()

To use this model, you would need to call the AddMatch method every time that you add an entity that would potentially be searched on. In your example, you have a Product model and users will be searching on it's Name. In your Product class, you might have a method AddNewProduct that creates a new entity and puts it into the datastore. You would add to that method StringIndex.AddMatch(new_product_name).

Then, in your request handler that gets called from your AJAXy search box, you would use StringIndex.GetMatchesFor(name) to see all of the stored products that begin with the string in name, and you return those values as JSON or whatever.

What's happening inside the code is that the first three characters of the name are used for the key_name of an entity that contains a list of strings, all of the stored names that begin with those three characters. Using three (as opposed to some other number) is absolutely arbitrary. The correct number for your system is dependent on the amount of data that you are indexing. There is a limit to the number of strings that can be stored in a StringListProperty, but you also want to balance the number of StringIndex entities that are in your datastore. A little bit of math with give you a reasonable number of characters to work with.

Upvotes: 1

Related Questions