CODEWITHSUNDEEP
mysqldjango
Jose M Martin
Jose M Martin

Reputation: 373

Django bulk_create IntegrityError (Duplicate entry) with UUID primary_key

I'm trying to save a huge amount of objects in a model, so I decided to use bulk_create by this way:

while True:
    batch = list(islice(records, 100))
    if not batch:
        break
    models.Record.objects.bulk_create(batch, batch_size)

But I get this error when I add the second batch of 100:

django.db.utils.IntegrityError: (1062, "Duplicate entry '165ab345f2ad47bbb2072ab7b3e8023f' for key 'PRIMARY'")

This is the model if it helps you:

class Record(models.Model):
    id = models.UUIDField(primary_key=True, default=uuid.uuid4, editable=False)
    ...

Thanks in advance.

Upvotes: 1

Views: 1641

Answers (1)

Rachel
Rachel

Reputation: 366

I believe your code is returning the same slice every time, so you get the integrity error because you've already inserted that particular record. The current code is starting from the same starting location. Try using standard python slicing instead.

record_list = []
for record in records:
    if len(record_list) % 100 == 0:
        models.Record.objects.bulk_create(record_list, len(record_list))
        record_list = []
    else:
        record_list.append(record)

According to islice documentation, your current code is returning all but the first 100 elements of your records as a single slice . See more here, https://docs.python.org/3.7/library/itertools.html#itertools.islice

Upvotes: 2

Related Questions