CODEWITHSUNDEEP
djangodjango-southdjango-migrations
yekta
yekta

Reputation: 3433

Do django db_index migrations run concurrently?

I'm looking to add a multi-column index to a postgres database. I have a non blocking SQL command to do this which looks like this:

CREATE INDEX CONCURRENTLY shop_product_fields_index ON shop_product (id, ...);

When I add db_index to my model and run the migration, will it also run concurrently or will it block writes? Is a concurrent migration possible in django?

Upvotes: 21

Views: 10035

Answers (6)

kmmbvnr
kmmbvnr

Reputation: 6139

There is no support for PostgreSQL concurent index creation in django.

Here is the ticket requesting this feature - https://code.djangoproject.com/ticket/21039

But instead, you can manually specify any custom RunSQL operation in the migration - https://docs.djangoproject.com/en/stable/ref/migration-operations/#runsql

Upvotes: 0

tgroshon
tgroshon

Reputation: 316

With Django 1.10 migrations you can create a concurrent index by using RunSQL and disabling the wrapping transaction by making the migration non-atomic by setting atomic = False as a data attribute on the migration:

class Migration(migrations.Migration):
    atomic = False # disable transaction

    dependencies = []

    operations = [
        migrations.RunSQL('CREATE INDEX CONCURRENTLY ...')
    ]

Upvotes: 14

anjaneyulubatta505
anjaneyulubatta505

Reputation: 11675

You can do something like


import django.contrib.postgres.indexes
from django.db import migrations, models
from django.contrib.postgres.operations import AddIndexConcurrently


class Migration(migrations.Migration):

    atomic = False

    dependencies = [
        ("app_name", "parent_migration"),
    ]

    operations = [
        AddIndexConcurrently(
            model_name="mymodel",
            index=django.contrib.postgres.indexes.GinIndex(
                fields=["field1"],
                name="field1_idx",
            ),
        ),
        AddIndexConcurrently(
            model_name="mymodel",
            index=models.Index(
                fields=["field2"], name="field2_idx"
            ),
        ),
    ]

Ref: https://docs.djangoproject.com/en/dev/ref/contrib/postgres/operations/#django.contrib.postgres.operations.AddIndexConcurrently

Upvotes: 1

Max Malysh
Max Malysh

Reputation: 31585

There are AddIndexConcurrently and RemoveIndexConcurrently in Django 3.0:

https://docs.djangoproject.com/en/dev/ref/contrib/postgres/operations/#django.contrib.postgres.operations.AddIndexConcurrently

Create a migration and then change migrations.AddIndex to AddIndexConcurrently. Import it from django.contrib.postgres.operations.

Upvotes: 18

Eugene Yarmash
Eugene Yarmash

Reputation: 149823

You could use the SeparateDatabaseAndState migration operation to provide a custom SQL command for creating the index. The operation accepts two lists of operations:

  • state_operations are operations to apply on the Django model state. They do not affect the database.

  • database_operations are operations to apply to the database.

An example migration may look like this:

from django.db import migrations, models

class Migration(migrations.Migration):
    atomic = False

    dependencies = [
        ('myapp', '0001_initial'),
    ]

    operations = [    
        migrations.SeparateDatabaseAndState(    
            state_operations=[
                # operation generated by `makemigrations` to create an ordinary index
                migrations.AlterField(
                    # ...  
                ),
            ],

            database_operations=[
                # operation to run custom SQL command (check the output of `sqlmigrate`
                # to see the auto-generated SQL, edit as needed)
                migrations.RunSQL(sql='CREATE INDEX CONCURRENTLY ...',
                                  reverse_sql='DROP INDEX ...'),
            ],
        ),
    ]

Upvotes: 11

lee penkman
lee penkman

Reputation: 1238

Do what tgroshon says for new django 1.10 +

for lesser versions of django i have had success with a more verbose subclassing method:

from django.db import migrations, models


class RunNonAtomicSQL(migrations.RunSQL):
    def _run_sql(self, schema_editor, sqls):
        if schema_editor.connection.in_atomic_block:
            schema_editor.atomic.__exit__(None, None, None)
        super(RunNonAtomicSQL, self)._run_sql(schema_editor, sqls)


class Migration(migrations.Migration):
    dependencies = [
    ]

    operations = [

        RunNonAtomicSQL(
            "CREATE INDEX CONCURRENTLY",
        )
    ]

Upvotes: 1

Related Questions