CODEWITHSUNDEEP
pythongoogle-app-enginenosqlgoogle-cloud-datastore
sahid
sahid

Reputation: 2610

Avoiding datastore contention

I have a design problem, my app has a grows up traffic and I have a lot a "Transaction collision. Retrying...". I don't know how avoiding them.

Introduction: Each Actor in this app can visit an another Actor.

That is my actual design (it can have errors, it just an example):

class Actor(object):
  name = db.StringProperty()

class ActorVisits(object):
  """Store a list of all actors has visited the parent of this kind

  Note: Has an ancestor an Actor
  """
  visits = db.ListProperty(db.Key)

class Visit(object):
  """A new visit by actor on the parent of this kind.
  Each time a same actor visits the parent of this kind, this kind is updated
  with the new datetime for the visit.

  Note: Has an ancestor an Actor
  """
  actor = db.ReferenceProperty(collection_name=Actor)
  visited_at = db.datetimeProperty()

That is an example of use case:

foo = Actor(name="foo")  
bar = Actor(name="bar")

db.put([foo, bar])

def trx():
  """Store a new visite from 'bar' to 'foo' """

  av = ActorVisits.get_bey_keys_name(key_name="v/foo", parent=foo)
  if av is None:
    av = ActorVisits(key_name="v/foo", parent=foo)

  v = Visit.get_bey_keys_name(key_name="v/foo/bar", parent=foo, actor=bar)
  if v is None:
    v = Visit(key_name="v/foo/bar", parent=foo, actor=bar)
  v.visited_at = datetime.datetime.utcnow()
  async = db.put_async(v)

  if not v.key() in av.visits:
    av.visits.append(v.key())
    av.put()

  async.get_result()

db.run_in_transaction(trx, bar, foo)

If anybody has an idea for me to make this model better. Thanks

Upvotes: 1

Views: 417

Answers (1)

aidanok
aidanok

Reputation: 859

Make the Visits eventually consistent, by moving them out of the Actor's EntityGroup. If each Visit is a root entity, you wont run into any write rate issues, and you can query for them easily.

If you don't want to create lots of Visit Entities, but just update a single one, you can make the key of the Visit deterministic, by basing it off the Actor, and the visitor.

So Actor has Id of 123, Vistor(another Actor) has Id of 888,

You make a key of type Visit, with Id of "123_888" (no parent) , You put() the Visit with this key, and updated time, and it overwrites previous Visit. You can also do a get() on the key, and it will be strongly consistent (if you are keeping a count say).

You can then build ActorVisits with an eventually consistent query.

Upvotes: 2

Related Questions