CODEWITHSUNDEEP
pythonmysqldjangotransactions
Joddy
Joddy

Reputation: 2695

Django 1.6 Transaction Management on large list of Model Objects

Long question short : If I would need to loop through list of Model Objects, and save them individually, I am unsure how to go about that.

To understand this correctly,

Assuming, my Model Name = Foo

example :

Model Object :
f1 = Foo 1 {
name : foo,
alias : Fooo 1,
description : Fooo 1
}

f2 = Foo 2 {
name : foo_2,
alias : Fooo,
description : Fooo 2
}

f3 = Foo 3 {
name : foo_3,
alias : Fooo 3,
description : Fooo
}

Now what I would like to do is change :
name for (Foo 1)
alias for (Foo 2)
description for (Foo 3)
and then manually do the transaction of updating (saving) each object via looping.

foo_list = list()

f1.name = 'foo_1'
foo_list.append(f1)

f2.alias = 'Foo 2'
foo_list.append(f2)

f3.description = 'Fooo 3'
foo_list.append(f3)
@task()
@transaction.atomic()
def bulk_update_decorated(bulky):
    """

    :param bulky: model object list
    :return: True
    """
    sid = None
    try:

        for update_this in bulky:
            update_this.save()            # transaction is having an element
            sid = transaction.savepoint()
        return True
    except SoftTimeLimitExceeded:
        transaction.savepoint_commit(sid) # on loop exit commit

@task()
def bulk_update_internal(bulky):
    """

    :param bulky: model object list
    :return: True
    """
    sid = None
    try:
        with transaction.atomic():
            for update_this in bulky:
                update_this.save()
                sid = transaction.savepoint()
    except SoftTimeLimitExceeded:
        transaction.savepoint_commit(sid) # on loop exit commit

Which of the following is the preferred way to go (actually which might work) ?

bulk_update_internal(foo_list)

OR

bulk_update_decorated(foo_list)

................................................

I am having quite a large set of data which I want to save/update.

Earlier I was looping through and saving them, which did not seem like a good idea.

Can't use update(), because I want each data point to be storing different information and saved separately.

Edit : Intended use

What I intended to do was, via using CELERY, update the list of model objects . So I was looping over the list like

for update_this in bulky: update_this.save()

This would mean save method be called for all the objects, and since Django 1.6 is autocommit, this would mean objects are saved in database in a loop. Bad Idea.

So, using transaction.atomic() would help me here right ?
It would start with autocommit = 0 and on loop exit would commit to database ?

for celery i wanted to use sid to hold objects before soft time limit would be reached. This would be a good strategy ?

Upvotes: 0

Views: 219

Answers (1)

catavaran
catavaran

Reputation: 45575

Use the bulk_update_decorated version. But as far as I understand you don't need to do something with savepoints. Just return from the function and whole transaction will be committed.

@transaction.atomic
def bulk_update_decorated(bulky):
    try:
        for update_this in bulky:
            update_this.save()
    except SoftTimeLimitExceeded:
        pass

Savepoints are used to partial rollback of transaction. You don't do any rollbacks so using of savepoint makes no sense here.

Upvotes: 1

Related Questions