CODEWITHSUNDEEP
django
Crazy Frog
Crazy Frog

Reputation: 525

Django - copy and insert queryset clone using bulk_create

My goal is to create a clone of a queryset and then insert it into the database. Following the suggestions of this post, I have the following code:

qs_new = copy.copy(qs)
MyModel.objects.bulk_create(qs_new)

However, with this code I run into duplicate primary key error. As for now, I only can come up with the following work-around:

qs_new = copy.copy(qs)
for x in qs_new:
   x.id = None
MyModel.objects.bulk_create(qs_new)

Question: Can I implement this code snippet without going through loop ?

Upvotes: 4

Views: 3707

Answers (3)

Robin LI
Robin LI

Reputation: 61

I tried and you need a loop to set the id to None, then it works. so finally it may be like this:

    qs_new = copy.copy(qs)
    for q in qs_new:
        q.id = None
        # also, you can set other fields if you need
    MyModel.objects.bulk_create(qs_new)

This works for me.

Upvotes: 1

Risadinha
Risadinha

Reputation: 16666

Note that bulk_create behaves differently depending on the underlying database. With Postgres you get the new primary keys set:

Support for setting primary keys on objects created using bulk_create() when using PostgreSQL was added. https://docs.djangoproject.com/en/1.10/ref/models/querysets/#django.db.models.query.QuerySet.bulk_create

You should, however make sure that the objects you are creating either have no primary keys or only keys that are not taken yet. In the latter case you should run the code that sets the PKs as well as the bulk_create inside transaction.atomic().

Fetching the values explicitly as suggested by Shang Wang might be faster because only the given values are retrieved from the DB instead of fetching everything. If you have foreign key relations or m2m relations you might want to avoid simply throwing the complex instances into bulk_create but instead explicitly naming all attributes that are required when constructing a new MyModel instance.

Here an example:

class MyModel(Model):
    name = TextField(...)
    related = ForeignKeyField(...)
    my_m2m = ManyToManyField(...)

In case of MyModel above, you would want to preserve the ForeignKey relations by specifying related_id and the PK of the related object in the constructor of MyModel, avoiding specifying related.

With m2m relations, you might end up skipping bulk_create altogether because you need each specific new PK, the corresponding original PK (from the instance that was copied) and the m2m relations of that original instance. Then you would have to create new m2m relations with the new PK and these mappings.

# add all fields here except 'id'
qs = qs.values('name', 'related_id')
MyModel.objects.bulk_create([MyModel(**i) for i in qs])

Note for completeness: If you have overriden save() on your model (or if you are inheriting from 3rd party with custom save methods), it won't be executed and neither will any post_save handlers (yours or 3rd party).

Upvotes: 0

Shang Wang
Shang Wang

Reputation: 25539

Can't think of a way without loop, but just a suggestion:

# add all fields here except 'id'
qs = qs.values('field1', 'field2', 'field3')
new_qs = [MyModel(**i) for i in qs]
MyModel.objects.bulk_create(new_qs)

Upvotes: 7

Related Questions