CODEWITHSUNDEEP
javagoogle-app-enginegoogle-cloud-datastore
crazystick
crazystick

Reputation: 610

How does AppEngine count entity groups for a transaction?

If I have the following set of entities:

A --> (unowned) B
  \
   --> (owned) List<C>

D --> (owned) List<E> --> (owned) List<F> --> (unowned) A
  \
   --> (unowned) H

G --> (unowned) H
  \
   --> (unowned) D
  \
   --> (unowned) B
  \
   --> (unowned) A
  \
   --> (unowned) F
  \
   --> (unowned) F

If I'm touching all of these objects in a transaction, I count 4 entity groups (A, B, D, H) This should be allowed (according to the docs, you can have up to 5).

So, 2 questions: 1) Does it matter how you get/create them? i.e. is

C c = new C(a);
a.getCs().add(c);

somehow using two separate entity groups even though its a parent/child relationship? Or the fact that G has two distinct values of F -- is that two entity groups or one?

2) How deep does it go when you access an object? i.e. if I also have 

D --> (owned) List<K> --> (owned) List<L> --> (unowned) M

does appengine include M in the list of entity groups accessed in the transaction, even though I'm not accessing K, L or M?

I'm fairly happy with my object model from a conceptual point of view (without appengine I'm pretty sure I'd design it the same way again) but should I do as others have suggested on here and create a GenericObject which is somehow the parent of everything?

Or alternatively, given that this is pretty trivial in the database world, how many have abandoned the DataStore for Cloud SQL 6 months in? (perhaps that last is too subjective for stackoverflow, but it's a genuine question)

UPDATE

After wading through the logs generated by turning everything up to debug, I can see that at some point I get the following line:

24634 [1419746043@qtp-647750325-2] DEBUG DataNucleus.Persistence - Performing reachability algorithm on object with id "com.google.appengine.api.datastore.Key:D("[email protected]")"

which is then followed by a whole ton of retrievals of every object which can be accessed starting at that root entity. Since this includes a bunch of entities which are unowned (that is, they don't use any of the entities accessed so far as their parent) I guess this is what causes my transaction to fail, and the answer to Q2 appears to be 'everywhere'

So...Q3 - how do I prevent this behaviour. Note that I am calling makePersistent(D) in order to persist the two instances of F which have been modified.

Upvotes: 2

Views: 786

Answers (2)

crazystick
crazystick

Reputation: 610

I have managed to solve the problem by adding the following to jdoconfig.xml:

<property name="datanucleus.persistenceByReachabilityAtCommit" value="false"/>

This is documented at http://www.datanucleus.org/products/datanucleus/performance_tuning.html

If anyone would like to leave me a comment about why this is necessary, or possibly why my object model is wrong, I'd very much appreciate it, as I'm new to JDO and appengine.

Upvotes: 2

Peter Knego
Peter Knego

Reputation: 80340

A few notes:

  1. All high-level Datastore APIs (JDA/JPA/Objectify) are building upon low-level api. They can not do "more" than low-level API does. If you want to understand Datastore you should understand low-level API.

  2. The relationships between entities are based on one entity having a property with a key of another entity. Simply said: entity contains ID of another entity.

  3. You can broke entity relationship by simply removing key property from entity. There is no referential integrity as you would be used in SQL world: you can delete entity on either side of relationship with no constraints.

  4. Entity groups are based on ancestor relationship and HAVE NOTHING TO DO with entity relationships mentioned above in point 2. Ancestor relationship is based on keys: a child key will contain keys of all parent keys.

  5. Since ancestor relationship is based on keys it can only be established when entity is created. It can not be changed later as keys are immutable.

  6. Entity groups have a write/update limit of 1 write/s. So putting all entities under a one common GenericObject is really a bad idea. You should really be careful how you design Entity Groups as they will affect performance. Good starting point is putting each user entity as the root of Entity Group.

  7. Entity groups were designed to define transaction scope (basically it means that those entities reside on the same server, hence the write limit) and you SHOULD NOT use them to make logical relationships between entities.

Upvotes: 3

Related Questions