CODEWITHSUNDEEP
pythondjango
Nikolay Fominyh
Nikolay Fominyh

Reputation: 9256

Django. Thread safe update or create.

We know, that update - is thread safe operation. It means, that when you do:

  SomeModel.objects.filter(id=1).update(some_field=100)

Instead of:

sm = SomeModel.objects.get(id=1)
sm.some_field=100
sm.save()

Your application is relativly thread safe and operation SomeModel.objects.filter(id=1).update(some_field=100) will not rewrite data in other model fields.

My question is.. If there any way to do 

  SomeModel.objects.filter(id=1).update(some_field=100)

but with creation of object if it does not exists?

Upvotes: 15

Views: 11974

Answers (6)

wuyazi
wuyazi

Reputation: 1

SomeModel.objects.filter(id=1).update(set__some_field=100)

Upvotes: -3

GDorn
GDorn

Reputation: 8811

You want django's select_for_update() method (and a backend that supports row-level locking, such as PostgreSQL) in combination with manual transaction management.

try:
    with transaction.commit_on_success():
        SomeModel.objects.create(pk=1, some_field=100)
except IntegrityError: #unique id already exists, so update instead
    with transaction.commit_on_success():
        object = SomeModel.objects.select_for_update().get(pk=1)
        object.some_field=100
        object.save()

Note that if some other process deletes the object between the two queries, you'll get a SomeModel.DoesNotExist exception.

Django 1.7 and above also has atomic operation support and a built-in update_or_create() method.

Upvotes: 4

feifan.overflow
feifan.overflow

Reputation: 565

I think if you have critical demands on atom operations. You'd better design it in database level instead of Django ORM level.

Django ORM system is focusing on convenience instead of performance and safety. You have to optimize the automatic generated SQL sometimes.

"Transaction" in most productive databases provide database lock and rollback well.

In mashup(hybrid) systems, or say your system added some 3rd part components, like logging, statistics. Application in different framework or even language may access database at the same time, adding thread safe in Django is not enough in this case.

Upvotes: 0

Nik
Nik

Reputation: 438

from django.db import IntegrityError

def update_or_create(model, filter_kwargs, update_kwargs)
    if not model.objects.filter(**filter_kwargs).update(**update_kwargs):
        kwargs = filter_kwargs.copy()
        kwargs.update(update_kwargs)
        try:
            model.objects.create(**kwargs)
        except IntegrityError:
            if not model.objects.filter(**filter_kwargs).update(**update_kwargs):
                raise  # re-raise IntegrityError

I think, code provided in the question is not very demonstrative: who want to set id for model? Lets assume we need this, and we have simultaneous operations:

def thread1():
    update_or_create(SomeModel, {'some_unique_field':1}, {'some_field': 1})

def thread2():
    update_or_create(SomeModel, {'some_unique_field':1}, {'some_field': 2})

With update_or_create function, depends on which thread comes first, object will be created and updated with no exception. This will be thread-safe, but obviously has little use: depends on race condition value of SomeModek.objects.get(some__unique_field=1).some_field could be 1 or 2.

Django provides F objects, so we can upgrade our code:

from django.db.models import F

def thread1():
    update_or_create(SomeModel, 
                     {'some_unique_field':1}, 
                     {'some_field': F('some_field') + 1})

def thread2():
    update_or_create(SomeModel, 
                     {'some_unique_field':1},
                     {'some_field': F('some_field') + 2})

Upvotes: 7

dswistowski
dswistowski

Reputation: 171

It's impossible in django do such upsert operation, with update. But queryset update method return number of filtered fields so you can do:

from django.db import router, connections, transaction

class MySuperManager(models.Manager):
     def _lock_table(self, lock='ACCESS EXCLUSIVE'):
         cursor = connections[router.db_for_write(self.model)]
         cursor.execute(
            'LOCK TABLE %s IN %s MODE' % (self.model._meta.db_table, lock)
        )

     def create_or_update(self, id, **update_fields): 
         with transaction.commit_on_success():            
             self.lock_table()
             if not self.get_query_set().filter(id=id).update(**update_fields):
                self.model(id=id, **update_fields).save()

this example if for postgres, you can use it without sql code, but update or insert operation will not be atomic. If you create a lock on table you will be sure that two objects will be not created in two other threads.

Upvotes: 0

Jordan
Jordan

Reputation: 32552

You can use Django's built-in get_or_create, but that operates on the model itself, rather than a queryset.

You can use that like this:

me = SomeModel.objects.get_or_create(id=1)
me.some_field = 100
me.save()

If you have multiple threads, your app will need to determine which instance of the model is correct. Usually what I do is refresh the model from the database, make changes, and then save it, so you don't have a long time in a disconnected state.

Upvotes: 0

Related Questions