CODEWITHSUNDEEP
google-cloud-datastoredatastore
ffejrekaburb
ffejrekaburb

Reputation: 731

Google Cloud Datastore - transaction

I have a procedure run on a cron schedule which has equivalently the following steps as part of a google cloud app. #clear Kind1 for p in model.Kind1.query(): p.key.delete()

#repopulate Kind1 with date from Kind2
for p1 in model.Kind2.query():
  a = new Kind1()
  a.key = p1.key
  model.Kind1.put(a)

As you read you see Kind1 is temporarily blank.

Is there a built in way inside of Google Datastore (or another part of the environment) to create a transaction from this procedure to create a form of lock? I am concerned the procedure as written can create unavailability for a period of time.

Another thought to find the sources using Kind1 and create a cache that is invalidated upon completion of the above procedure.

Are there any other recommendations for how to handle this?

Upvotes: 0

Views: 59

Answers (1)

Joshua Melcon
Joshua Melcon

Reputation: 249

Non ancestor queries are eventually consistent in Cloud Datastore which makes this problem more challenging. Eventually this will change when the Firestore upgrade happens. Recall that eventual consistency means that the results of non ancestor queries may be out of date by an arbitrarily and unknowable amount of time.

To get around the eventual consistency problem you would have to put the entities into a common ancestor and then query that. That group would have a write rate limit of about 1 write per second -- if you have to write the collection faster than that you would need to add additional ancestor shards.

Once you are using consistent queries this is one solution:

  1. Create a new instance of the Kind1 collection to be queried under a new name. Populate it by querying Kind2.
  2. Whatever is querying this collection reads another entity and uses it to determine what query to use -- you can update that entity to point at the new name. Have the clients read that entity to decide what collection to query (you can probably stick this entity into memcache and refresh it periodically).

Another option is to create an appropriate composite index that allows you to query Kind2 directly without having to copy it over.

Upvotes: 1

Related Questions