CODEWITHSUNDEEP
python-3.xgoogle-app-enginegoogle-cloud-datastore
Noortheen Raja
Noortheen Raja

Reputation: 861

GAE Datastore client: How to create enforce unique constraint on properties

I am using anom to map with GAE datastore entities. I am searching for a scalable way to implement unique constraint on properties.

So far I am checking with queries before creating new records like this

class User(Model):
    ...
    email_id = props.String(indexed=True)
    ...

    def pre_put_hook(self):
        if (not self.id_or_name) and (User.query().where(User.email_id == self.email_id).count() > 0):
            raise IntegrityError(f'Duplicate entry: User.email == {self.email_id}')

It doesn't handle race conditions and will not force uniqueness on db level. Is there any way to do it without any race conditions?

Upvotes: 1

Views: 1136

Answers (3)

Noortheen Raja
Noortheen Raja

Reputation: 861

I have found this snippet from webapp2's repository and modified it for anom with a mixin to easy integration into any number of models. Now I can confirm the uniqueness of multiple fields inside a single entity. Of-course this suffers extra reads when creating new records and these fields are not checked for uniqueness when updating. So these fields must not be allowed to be updated or a similar check can be done when updating these fields in particular. Here is the gist. I hope this will be useful when using new Python 3 runtime of GAE and anom as data-mapper

Upvotes: 0

Dan Cornilescu
Dan Cornilescu

Reputation: 39824

The reason for which your current method is still prone to race conditions is the fact that (non-ancestor) query results are always eventually consistent. Even if you use ancestor queries (the only queries allowed inside a transaction) you may still probably be facing race conditions due to the datastore's Isolation and consistency model.

As Alex mentions, the only way to ensure uniqueness would be to use the property value as the entity's key identifier (you'd be checking the key's existence in your pre_put_hook method, see TypeError with get_or_insert). But you can only do that for a single property for an entity kind and in particular for the email address property it'd not be a great idea as it'd be difficult to handle email address changes.

There could be ways of enforcing uniqueness, but allowing a temporary/transient potentially duplicate condition, see ndb verify entity uniqueness in transaction.

Upvotes: 1

Alex
Alex

Reputation: 5276

There's no great way to do it at the db level besides using 'email' as your primary key do user. You'll be relying on your web application code / ORM (whether it be anom or ndb) to do the enforcement.

See this answer

composite key in google cloud datastore

Upvotes: 2

Related Questions