Automatically generate custom migrations in Django

Question

Django provides a really nice feature called makemigrations where it will create migration files based on the changes in models. We are developing a module where we will want to generate custom migrations.

I haven't found much info about creating custom migrations in the Django docs. There is documentation on the various Operation classes that can yield migrations but there's nothing about creating custom Operation classes that can yield custom migrations.

The autodetector module for generating new migrations also doesn't seem to leave much room for adding custom Operation classes: https://github.com/django/django/blob/master/django/db/migrations/autodetector.py#L160

It seems this is completely static. Are there other ways to generate custom migrations, perhaps by using existing classes with a custom management command?

Answer 1

You can create a custom class to hook into the makemigrations class and add your custom migrations stuff then execute using the "runscript" command. Below is a sample module where the file is named custom_migrations.py and located in a "scripts" folder off one of your apps:

from django.core.management.commands.makemigrations import Command
"""
To invoke this script use:
   manage.py runscript custom_migrations --script-args [app_label [app_label ...]] name=my_special_migration verbosity=1
"""    

class MyMigrationMaker(Command):
    '''
    Override the write method to add more stuff before finishing
    '''


    def write_migration_files(self, changes):
        print("Do some stuff to \"changes\" object here...")
        super().write_migration_files(changes)



def run(*args):
    nargs = []
    kwargs = {}
    # Preload some options with defaults and then can be overridden in the args parsing
    kwargs['empty'] = True
    kwargs['verbosity'] = 1
    kwargs['interactive'] = True
    kwargs['dry_run'] = False
    kwargs['merge'] = False
    kwargs['name'] = 'custom_migration'
    kwargs['check_changes'] = False
    for arg in args:
        kwarg = arg.split('=', 1)
        if len(kwarg) > 1:
            val = kwarg[1]
            if val == "True":
                arg_val = True
            elif val == "False":
                arg_val = False
            elif val.isdigits():
                arg_val = int(val)
            else:
                arg_val = val
            the_kwargs[kwarg[0]] = arg_val
        else:
            nargs.append(arg)
    MyMigrationMaker().handle(*nargs, **kwargs)

Answer 2

An alternative if you are not willing to Tango with Django internals is to use this script below to generate a migration file by invoking a random method that will produce the Python code you want to run in the migration and insert it into a valid migration file that will then be part of the standard Django migrations. It you had an app named "xxx" and you had a method in a file xxx/scripts/test.py looking like this:

def run(*args, **kwargs):
return "print(\"BINGO!!!!!!!!! {} :: {}\".format(args[0], kwargs['name']))"

... and you invoked the script shown at the bottom of this post stored in xxx/scripts/custom_migrations.py with the following command

manage.py runscript custom_migrations --script-args xxx name=say_bingo callable=xxx.scripts.test.run

Then you would end up with a migration file in xxx/migrations with the appropriate number sequence (something like 0004_say_bingo.py) looking like this:

 Generated by Django 2.1.2 on 2018-12-14 08:54

from django.db import migrations
def run(*args, **kwargs):
    print("BINGO!!!!!!!!! {} :: {}".format(args[0], kwargs['name']))

class Migration(migrations.Migration):

    dependencies = [
    ]

    operations = [
        migrations.RunPython(run,)
    ]

The script is as follows:

from django.core.management.base import no_translations
from django.core.management.commands.makemigrations import Command
from django.db.migrations import writer
from django import get_version
from django.utils.timezone import now
import os
import sys

"""
To invoke this script use:
    manage.py runscript custom_migrations --script-args [app_label [app_label ...]] callable=my_migration_code_gen name=my_special_migration

  -- the "name" argument will be set as part of the migration file generated
  -- the "callable" argument will be the function that is invoked to generate the python code you want to execute in the migration
    the callable will be passed all the args and kwargs passed in to this script from the command line as part of --script-args
    Only app names are allowed in args so use kwargs for custom arguments
"""


class LazyCallable(object):
    def __init__(self, name):
    self.n, self.f = name, None

    def __call__(self, *a, **k):
        if self.f is None:
            modn, funcn = self.n.rsplit('.', 1)
            if modn not in sys.modules:
                __import__(modn)
            self.f = getattr(sys.modules[modn], funcn)
        return self.f(*a, **k)


class MyMigrationMaker(Command):
    '''
    Override the write method to provide access to script arguments
    '''

    @no_translations
    def handle(self, *app_labels, **options):
        self.in_args = app_labels
        self.in_kwargs = options
        super().handle(*app_labels, **options)

    '''
    Override the write method to add more stuff before finishing
    '''

    def write_migration_files(self, changes):
        code = LazyCallable(self.in_kwargs['callable'])(self.in_args, self.in_kwargs)
        items = {
            "replaces_str": "",
            "initial_str": "",
        }
        items.update(
            version=get_version(),
            timestamp=now().strftime("%Y-%m-%d %H:%M"),
        )
        items["imports"] = "from django.db import migrations\n\ndef run(*args, **kwargs):\n    "
        items["imports"] += code.replace("\n", "\n    ") + "\n\n"
        items["operations"] = "        migrations.RunPython(run,)\n"
        directory_created = {}
        for app_label, app_migrations in changes.items():
            for migration in app_migrations:
                # Describe the migration
                my_writer = writer.MigrationWriter(migration)
                dependencies = []
                for dependency in my_writer.migration.dependencies:
                    dependencies.append("        %s," % my_writer.serialize(dependency)[0])
                    items["dependencies"] = "\n".join(dependencies) + "\n" if dependencies else ""

                # Write the migrations file to the disk.
                migrations_directory = os.path.dirname(my_writer.path)
                if not directory_created.get(app_label):
                    if not os.path.isdir(migrations_directory):
                        os.mkdir(migrations_directory)
                    init_path = os.path.join(migrations_directory, "__init__.py")
                    if not os.path.isfile(init_path):
                        open(init_path, "w").close()
                    # We just do this once per app
                    directory_created[app_label] = True
                migration_string =  writer.MIGRATION_TEMPLATE % items
                with open(my_writer.path, "w", encoding='utf-8') as fh:
                    fh.write(migration_string)
                if self.verbosity >= 1:
                    self.stdout.write("Migration file: %s\n" % my_writer.filename)



def run(*args):
    glob_args = []
    glob_kwargs = {}

    # Preload some options with defaults and then can be overridden in the args parsing
    glob_kwargs['empty'] = True
    glob_kwargs['verbosity'] = 1
    glob_kwargs['interactive'] = True
    glob_kwargs['dry_run'] = False
    glob_kwargs['merge'] = False
    glob_kwargs['name'] = 'custom_migration'
    glob_kwargs['check_changes'] = False
    for arg in args:
        kwarg = arg.split('=', 1)
        if len(kwarg) > 1:
            glob_kwargs[kwarg[0]] = kwarg[1]
        else:
            glob_args.append(arg)
    MyMigrationMaker().handle(*glob_args, **glob_kwargs)