CODEWITHSUNDEEP
pythonmysqldjangodjango-modelspersistence
Alexandre Cordeiro
Alexandre Cordeiro

Reputation: 439

How to force django to save immediately instead of doing a batch update after loop

I have this django views.py method that aims to insert many data into the db. It loops through arrays of models and, if an object isn't already on the db, it gets inserted.

This is what the code looks like:

def update_my_db(request):

    a_models = A_Model.objects.filter(my_flag=True)

    for a_model in a_models:

        b_model_array = []

        [...] # this is where b_model_array gets filled

        for index in range(len(b_model_array)):

            current_b_model = b_model_array[index]

            try:
                b_model = B_Model.objects.get(my_field=current_b_model.my_field)
            except (KeyError, B_Model.DoesNotExist):
                b_model = B_Model.objects.create(field_1=current_b_model.field_1, field_2=current_b_model.field_2)
                b_model.save()

    return HttpResponse(response)

I have noticed after several tests that the db is only updated by the end of the last iteration, as if django awaits to do a batch insert to mysql.

The thing is: there is a possibility of any of the iterations raising an exception, making all the data gathered so far be discarded because of the error (already tested and confirmed it). When it comes to adding 400 new lines, raising an exception at loop #399 and discarding all the previous 398 lines would be extremely undesirable for me.

I understand that batching would be the best choice concerning performance, but this is a background routine, so I'm not worried about it.

Bottomline: is there a way to actually force django to update the database on every iteration?

Upvotes: 4

Views: 2421

Answers (1)

Tim Wilder
Tim Wilder

Reputation: 1647

If you're on Django 1.6, check this out: https://docs.djangoproject.com/en/dev/topics/db/transactions/

You're interested in the context manager part of that page:

from django.db import transaction

def viewfunc(request):
    # This code executes in autocommit mode (Django's default).
    do_stuff()

    with transaction.atomic():
        # This code executes inside a transaction.
        do_more_stuff()

Upvotes: 3

Related Questions