CODEWITHSUNDEEP
pythongoogle-app-enginegoogle-cloud-datastore
user3848394
user3848394

Reputation: 67

Hierarchical queries on datastore

I am trying to create hierarchical queries on the App Engine.

My datastore have parents and children. Each parent has children. Imagine that I have to find children. I have a condition on the parent and an other on the children for example, imagine a real family datastore, my conditions are: I want all children that are a boy from parent who are 35 years old or more.

The query I have for now is something like: 

P = Parent.query(Parent.age >= 35)
for p in P:
    C = Children.query(gender == "boy", ancestor = p.key)
    for c in C:
        -> here I print information on the children

But the query is very slow with a lot of parent and children to test. I want to avoid iteration like for etc. because I think it take a long time because of that! What is the best practice to have his kind of query but quickly?

I have also brothers for the children and I can make queries, for example if I want all children that have parent that are more that 35 yers old and a sister named "Sisi", I will have (each children have his brothers on the value "brother"): 

 P = Parent.query(Parent.age >= 35)
        for p in P:
            C = Children.query(gender == "girl", name == "Sisi", ancestor = p.key)
            for c in C:
                C1 = Children.query(gender == "boy", brother == c.key, ancestor = p.key)
                for c1 in C1:
                    ->Here I print information about the children

In fact this example (family example) good for my project but it give an idea of the problem I have

Upvotes: 2

Views: 567

Answers (1)

Bardia D.
Bardia D.

Reputation: 693

How I've been able to do this before is to store the keys in a separate lookup entity. This goes along the key-value store premise that duplicate info is sometimes necessary for faster lookups. For example:

ParentChildLookup
 - parent_key = ndb.KeyProperty()
 - child_key = ndb.KeyProperty()

You could even add a third dimension of you are getting grandchildren:

ParentChildLookup
 - parent_key = ndb.KeyProperty()
 - child_key = ndb.KeyProperty()
 - grandchildren_key = ndb.KeyProperty()

If you wanted to look everything up in a single query, you can add the repeated clause to make children and grandchildren a list:

ParentChildLookup
 - parent_key = ndb.KeyProperty()
 - child_key = ndb.KeyProperty(repeated=True)
 - grandchildren_key = ndb.KeyProperty(repeated=True)

You then need to insert/update these lookup values anytime there's a change in relationships. The benefit of this is that you can avoid a lot of queries, specifically nested or multi-property queries. If you don't like this approach, I would recommend taking a look at the "Relationship Model" explanation here: https://cloud.google.com/appengine/articles/modeling. You can store the relationships between many-to-many objects without the need to store them all in the same entity.

Upvotes: 1

Related Questions