Ignore all current migrations and start from current model state WITHOUT deleting the migration files/history

Question

Preface: Migrating from Oracle to Postgres with a VERY large, VERY old Django app.

Need the ability to start a new db from the current state of my models, ignoring 5+ years of migrations, BUT without deleting any of the migration files.

Answer 1

I'm assuming you want to "start fresh" when creating database in PostgreSQL, but still be able to use (or even update) your migrations on old Oracle database, if necessary. I can think of 3 options to achieve that:

Squashing the migrations

This can, to some extent, be handled by Django itself, but Django will not always be correct about everything and it will actually leave some things to fix them manually, like any RunPython operations.

By simply invoking ./manage.py squashmigrations, you can ask Django to create an alternative migration to all old migrations you've had before. If any of the previous migrations are applied on your database, Django will just ignore the squashed migration that tries to replace them and will continue applying the old ones, but when none of the migrations replaced by squashed one are applied, Django will ignore old migrations and just apply the squashed one, treating it as a quicker way to achieve the same database state.

Creating a replacement (squashed migration) for existing migrations manually

This solution comes with understanding of what actually happens under the hood when you're squashing migrations in Django and it's very similar to the squashing process, but achieved in a different way.

Under the hood, when you try to squash migrations, Django will create a migration that just contains all the steps from all previous migrations (with some optimizations applied, if possible) and will add a special field replaces in the Migration class inside, containing a list of old migrations that were squashed.

This means that you can create your own migration that should replace a list of other ones. This is an advanced usage of migration system, so proceed with care! To achieve that:

1. Make sure that your current model state is fully represented in Django migrations. To check that, run ./manage.py makemigrations YOUR_APP_NAME --dry-run and make sure it returns No changes detected in app 'YOUR_APP_NAME'.
2. Temporarily move all migrations that you want to replace outside of migrations directory, so Django is not aware of their existence
3. Run ./manage.py migrate YOUR_APP_NAME to generate a new migration that replaces the old ones
4. Add replaces = [...] attribute at the top of Migration class in newly created migration, replacing three dots with a list of all migrations this migration shall replace (to get that list, you can create a squashed migration beforehand just to copy the replaces field from it)
5. Make sure the new migration name doesn't conflict with any of the old ones. You can just change it's name now without any problems, even if this is the initial migration (names of the migrations doesn't matter, they just need to be referenced properly by other migrations when necessary, you can even get rid of the migration number)
6. Move old migrations back to your migrations directory
7. Make sure that migrations are still creating the same state of the database as before, by invoking ./manage.py makemigrations YOUR_APP_NAME --dry-run`.

Note that for this solution (as for automatically created squash migration) you need to have one replacement migration per consistent sequence (or graph) of migration files. That means if you for example want to replace migrations from 0002 to 0008, keep 0009 and replace 0010 to 0012, you will need 2 replacement migrations, one per the sequence range. Same goes for a "graph" of migrations (if you for example have two 0004 migrations and a merging migration 0005) accordingly.

For both of the above options, Django will use old migration files when at least one of the old migrations is already applied on your database and will use the new one if none of them is applied. Every migration created after the squash migration will be used in both cases.

Keeping a separate migration lists for your databases.

It is also possible to just separate both migration lists by keeping them in separate directories. Django allows you to overwrite the default location for migration files, per app basis, by using MIGRATION_MODULES setting. Note that the migrations will be completely separate, so you will need to manually keep them both in sync. When doing some changes in your app models, you will need to run ./manage.py makemigrations twice, changing MIGRATION_MODULES setting between the runs or you will have to copy the migration over from one location to another. I highly advise against this solution as it is very easy to mess up the synchronization between migration lists and it has no benefit from the other methods except of the situation when you for some reason need the old process when recreating the Oracle database from scratch (you still can use the old method, but you will have to invoke one of the old migrations manually in such case).

Answer 2

My solution was to use Django, and have it do it all for me.

1. create the database using a fake connection using all the variables from default. Changed USER to postgres since it's the only user that can access the database postgres and NAME to postgres for the previous reason. ('main' i called it, it's only used the once, to create the database)
2. figure out what the path of my app was, loop over all my apps and if the app was local to my path, create a temporary migrations_<random> folder inside the app path, and a __init__.py inside that. finally, update MIGRATION_MODULES telling django where to find the migration folder for said app.
3. call ./manage.py makemigrations, and make them. (remember this will use the migrations from MIGRATION_MODULES)
4. call ./manage.py migrate, and apply them. (remember this will use the migrations from MIGRATION_MODULES)
5. delete ALL migrations in the django_migrations table
6. (clean up) set MIGRATION_MODULES to it's original state, delete all the migrations_<random> folders created in all the apps.
7. call ./manage.py migrate --fake to "apply" all the existing migrations you have in the project

enjoy!

import inspect
import os
import tempfile
from django.apps import apps
from django.db import connection, connections
from contextlib import ExitStack
from django.conf import settings
from django.core.management import call_command
from django.db.migrations.recorder import MigrationRecorder


def create_database_and_models():

    settings.DATABASES['main'] = settings.DATABASES['default'].copy()
    settings.DATABASES['main']['USER'] = 'postgres'
    settings.DATABASES['main']['NAME'] = 'postgres'

    with connections['main'].cursor() as cursor:
        database_name = settings.DATABASES['default']['NAME']
        cursor.execute('CREATE DATABASE %s' % (database_name,))
        print('Create db %s' % (database_name))

    original_migration_modules = settings.MIGRATION_MODULES.copy()
    project_path = os.path.abspath(os.curdir)

    with ExitStack() as stack:
        # Find each app that belongs to the user and are not in the site-packages. Create a fake temporary
        # directory inside each app that will tell django we don't have any migrations at all.
        for app_config in apps.get_app_configs():
            module = app_config.module
            app_path = os.path.dirname(os.path.abspath(inspect.getsourcefile(module)))

            if app_path.startswith(project_path):
                temp_dir = stack.enter_context(tempfile.TemporaryDirectory(prefix='migrations_', dir=app_path))
                # Need to make this directory a proper python module otherwise django will refuse to recognize it
                open(os.path.join(temp_dir, '__init__.py'), 'a').close()
                settings.MIGRATION_MODULES[app_config.label] = '%s.%s' % (app_config.module.__name__,
                                                                          os.path.basename(temp_dir))

        # create brand new migrations in fake migration folders local to the app
        call_command('makemigrations')
        # call migrate as normal, this will just read those fake folders with migrations in them.
        call_command('migrate')

        # delete all the migrations applied, remember these where NOT the real migrations from the project
        MigrationRecorder.Migration.objects.all().delete()

    settings.MIGRATION_MODULES = original_migration_modules

    # Fake all the current migrations in the project
    call_command('migrate', fake=True)