CODEWITHSUNDEEP
google-cloud-datastore
Travis Reeder
Travis Reeder

Reputation: 41103

Does Google Cloud Datastore support atomic operations, for counters and such?

Does Google Cloud Datastore support atomic operations, for counters and such? I don't see anything about in the documentation and I noticed that updates are basically getting the whole object, updating a field, then sending the whole object back which seems like it would make atomic operations impossible. https://cloud.google.com/datastore/docs/concepts/entities#Datastore_Updating_an_entity

Upvotes: 5

Views: 2688

Answers (2)

tommed
tommed

Reputation: 1561

Pop your updates in an ordered queue (for example Google Pub/Sub) and have a single processor/service that is the only service to have write permissions. The advantage of this method is you can then chose the persistence platform based on the data held and querying needs rather than atomic/acid requirements.

A simpler (and cheaper) option is to use a Background Function which is triggered by Pub/Sub so a nominated serverless function receives messages from Pub/Sub and you just concentrate on writing the increment logic.

Upvotes: 2

Isaac
Isaac

Reputation: 788

Not directly, but you can simulate an atomic increment operation using transactions. In the simplest case, you use a single entity and increment with a transactional read+write. Such a transaction will fail if there is a concurrent increment, so the counter does not scale well. Instead, you can operate on n entities with keys in the form of [('Counter', 'MyCounterX'), ('Elt', 'k')] for some number k, and property 'Count':

To increment, pick a random number between 1 and n, and attempt a transactional read+write. If the key doesn't exist, write a new entity on the given key with count=1. If the transaction fails, you can retry with a new random number. You could also include logic for the application to increase n on the fly it it starts hitting contention frequently.

To retrieve the count, do an ancestor query using root [('Counter', 'MyCounterX')] with a projection on the Count property, and then add up all the counts.

You can see code which implements this in the second block here.

Upvotes: 7

Related Questions