CODEWITHSUNDEEP
google-app-enginegoogle-cloud-datastoreapp-engine-ndb
ChrisS
ChrisS

Reputation: 734

GAE ndb best practice to store large one to many relations

I'm searching for the best practice to store a large amount of Comment Entities which have a one to many relationship to another entity.

I read a lot about the limitations about the datastore and don't know how to solve this.

I can't store them as structured properties due to the 1MB Entity Limitation.

Also Guido van Rossum answered the question about repeated properties with "if you have more than 100-1000 values" do not use repeated properties. So repeated properties are no solution for my comments, too.

Final Question: What is the best practice to solve this problem? Are ancestors an opportunity?

Edit: In this question about ancestor or reference properties Nick Johnson mentioned that "Every entity with the same parent will be in the same entity group, and writes to entity groups are serialized, so using ancestors here will slow things down if you're writing multiple entities concurrently. Since all the entities in a group are 'owned' by the user that forms the root of the group in your instance, though, this shouldn't be a problem - and in fact, what you're doing is actually a recommended design pattern."

What exactly does " writing multiple entities concurrently mean" ? When different user comment at the same time to that entity?

Upvotes: 0

Views: 497

Answers (1)

Jimmy Kane
Jimmy Kane

Reputation: 16825

Depends on the amount you read / write per bill.

You can store references for more than 1000 (until an amount depending by the key size and how you reference them) as json compressed unindexed properties. But take care then with referencing and dereferecing that amount. Plus your overhead and data amount that you will transfer on each request will be big. You don't want though to be doing ops on 1000000 compressed entity keys on the server for just a simple request. If you take this way trying to optimize this approach do it on the client as smart as you can.

Go for ancestors and/or optimize your logic not to be consistent (eg it doesn't matter if a comment is not shown immediately) and use iterators or pointer or seeks (whatever it's called)

Upvotes: 2

Related Questions