CODEWITHSUNDEEP
pythondjangopostgresqldjango-models
Paul
Paul

Reputation: 1242

"django.db.utils.ProgrammingError: relation "app_user" does not exist" during manage.py test

My setup:

I get the following error when I run python manage.py test:

Traceback (most recent call last):
  File "/src/venv/bin/django-admin.py", line 5, in <module>
    management.execute_from_command_line()
  File "/src/venv/lib/python2.7/site-packages/django/core/management/__init__.py", line 338, in execute_from_command_line
    utility.execute()
  File "/src/venv/lib/python2.7/site-packages/django/core/management/__init__.py", line 330, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/src/venv/lib/python2.7/site-packages/django/core/management/commands/test.py", line 30, in run_from_argv
    super(Command, self).run_from_argv(argv)
  File "/src/venv/lib/python2.7/site-packages/django/core/management/base.py", line 393, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/src/venv/lib/python2.7/site-packages/django/core/management/commands/test.py", line 74, in execute
    super(Command, self).execute(*args, **options)
  File "/src/venv/lib/python2.7/site-packages/django/core/management/base.py", line 444, in execute
    output = self.handle(*args, **options)
  File "/src/venv/lib/python2.7/site-packages/django/core/management/commands/test.py", line 90, in handle
    failures = test_runner.run_tests(test_labels)
  File "/src/venv/lib/python2.7/site-packages/django/test/runner.py", line 210, in run_tests
    old_config = self.setup_databases()
  File "/src/venv/lib/python2.7/site-packages/django/test/runner.py", line 166, in setup_databases
    **kwargs
  File "/src/venv/lib/python2.7/site-packages/django/test/runner.py", line 370, in setup_databases
    serialize=connection.settings_dict.get("TEST", {}).get("SERIALIZE", True),
  File "/src/venv/lib/python2.7/site-packages/django/db/backends/base/creation.py", line 368, in create_test_db
    test_flush=not keepdb,
  File "/src/venv/lib/python2.7/site-packages/django/core/management/__init__.py", line 120, in call_command
    return command.execute(*args, **defaults)
  File "/src/venv/lib/python2.7/site-packages/django/core/management/base.py", line 444, in execute
    output = self.handle(*args, **options)
  File "/src/venv/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 179, in handle
    created_models = self.sync_apps(connection, executor.loader.unmigrated_apps)
  File "/src/venv/lib/python2.7/site-packages/django/core/management/commands/migrate.py", line 317, in sync_apps
    cursor.execute(statement)
  File "/src/venv/lib/python2.7/site-packages/django/db/backends/utils.py", line 65, in execute
    return self.cursor.execute(sql, params)
  File "/src/venv/lib/python2.7/site-packages/django/db/utils.py", line 97, in __exit__
    six.reraise(dj_exc_type, dj_exc_value, traceback)
  File "/src/venv/lib/python2.7/site-packages/django/db/backends/utils.py", line 63, in execute
    return self.cursor.execute(sql)
django.db.utils.ProgrammingError: relation "app_user" does not exist

When I drop a print(sql) statement on line 62 in django/db/backends/utils.py, I get following output:

CREATE DATABASE "test_dev"

            SELECT c.relname, c.relkind
            FROM pg_catalog.pg_class c
            LEFT JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace
            WHERE c.relkind IN ('r', 'v')
                AND n.nspname NOT IN ('pg_catalog', 'pg_toast')
                AND pg_catalog.pg_table_is_visible(c.oid)
CREATE TABLE "django_migrations" ("id" serial NOT NULL PRIMARY KEY, "app" varchar(255) NOT NULL, "name" varchar(255) NOT NULL, "applied" timestamp with time zone NOT NULL)

            SELECT c.relname, c.relkind
            FROM pg_catalog.pg_class c
            LEFT JOIN pg_catalog.pg_namespace n ON n.oid = c.relnamespace
            WHERE c.relkind IN ('r', 'v')
                AND n.nspname NOT IN ('pg_catalog', 'pg_toast')
                AND pg_catalog.pg_table_is_visible(c.oid)
SAVEPOINT "s140275211773760_x1"
CREATE TABLE "distributedlock_lock" ("id" serial NOT NULL PRIMARY KEY, "key" varchar(255) NOT NULL, "value" varchar(255) NOT NULL, "timestamp" timestamp with time zone NULL)
RELEASE SAVEPOINT "s140275211773760_x1"
SAVEPOINT "s140275211773760_x2"
CREATE TABLE "djkombu_queue" ("id" serial NOT NULL PRIMARY KEY, "name" varchar(200) NOT NULL UNIQUE)
RELEASE SAVEPOINT "s140275211773760_x2"
SAVEPOINT "s140275211773760_x3"
CREATE TABLE "djkombu_message" ("id" serial NOT NULL PRIMARY KEY, "visible" boolean NOT NULL, "sent_at" timestamp with time zone NULL, "payload" text NOT NULL, "queue_id" integer NOT NULL)
RELEASE SAVEPOINT "s140275211773760_x3"
SAVEPOINT "s140275211773760_x4"
CREATE TABLE "otp_static_staticdevice" ("id" serial NOT NULL PRIMARY KEY, "user_id" integer NOT NULL, "name" varchar(64) NOT NULL, "confirmed" boolean NOT NULL)
RELEASE SAVEPOINT "s140275211773760_x4"
SAVEPOINT "s140275211773760_x5"
CREATE TABLE "otp_static_statictoken" ("id" serial NOT NULL PRIMARY KEY, "device_id" integer NOT NULL, "token" varchar(16) NOT NULL)
RELEASE SAVEPOINT "s140275211773760_x5"
SAVEPOINT "s140275211773760_x6"
CREATE TABLE "otp_totp_totpdevice" ("id" serial NOT NULL PRIMARY KEY, "user_id" integer NOT NULL, "name" varchar(64) NOT NULL, "confirmed" boolean NOT NULL, "key" varchar(80) NOT NULL, "step" smallint NOT NULL CHECK ("step" >= 0), "t0" bigint NOT NULL, "digits" smallint NOT NULL CHECK ("digits" >= 0), "tolerance" smallint NOT NULL CHECK ("tolerance" >= 0), "drift" smallint NOT NULL, "last_t" bigint NOT NULL)
RELEASE SAVEPOINT "s140275211773760_x6"
CREATE INDEX "djkombu_queue_name_1c24e49fd475ad53_like" ON "djkombu_queue" ("name" varchar_pattern_ops)
ALTER TABLE "djkombu_message" ADD CONSTRAINT "djkombu_message_queue_id_12778caea7843dd_fk_djkombu_queue_id" FOREIGN KEY ("queue_id") REFERENCES "djkombu_queue" ("id") DEFERRABLE INITIALLY DEFERRED
CREATE INDEX "djkombu_message_46cf0e59" ON "djkombu_message" ("visible")
CREATE INDEX "djkombu_message_df2f2974" ON "djkombu_message" ("sent_at")
CREATE INDEX "djkombu_message_75249aa1" ON "djkombu_message" ("queue_id")
ALTER TABLE "otp_static_staticdevice" ADD CONSTRAINT "otp_static_staticdevice_user_id_39a61f1bd3ec970d_fk_app_user_id" FOREIGN KEY ("user_id") REFERENCES "ff_user" ("id") DEFERRABLE INITIALLY DEFERRED

So it is clear to me that my tests blow up while the test database is being setup. Specifically, the attempt to create a foreign key constraint between the otp_static_staticdevice table and my app's app_user table fails.

My immediate question is, why does django create the OTP table before my app's table? My assumption is that the OTP app is listed first in my INSTALLED_APPS. But this is not the case:

INSTALLED_APPS = [
    'django.contrib.auth',
    'django.contrib.contenttypes',
    'django.contrib.sessions',
    'django.contrib.sites',
    'django.contrib.messages',
    'django.contrib.staticfiles',
    'django.contrib.admin',
    'django.contrib.humanize',
    'app',
    ...
    'django_otp',
    'django_otp.plugins.otp_static',
    'django_otp.plugins.otp_totp',
    'two_factor',
    ...
]

Next, I look at django/core/management/commands/migrate.py, trying to find out how django determines its order for migrating apps.

Plopping a pdb.set_trace() statement on line 264 (https://github.com/django/django/blob/1.8.3/django/core/management/commands/migrate.py#L264) and looking to see what app_labels contains, I get:

set(['djangosaml2', 'django_ace', 'recurly', 'staticfiles', 'distributedlock', 'app_overrides', 'messages', 'django_otp', 'kombu_transport_django', 'otp_totp', 'compressor', 'otp_static', 'humanize', 'ajax_select', 'django_extensions', 'import_export', 'raven_compat', 'crispy_forms', 'emoji'])

This is as far as I have gotten before I decided to ask for help. Does anyone know how Django might end up not creating all the project's apps in the correct order so that decency conflicts do not occur?

Upvotes: 10

Views: 12135

Answers (8)

ANFAS PV
ANFAS PV

Reputation: 453

Ensure all your app directories have migrations/ folder and __init__.py file in it. If not, make a migrations/ folder and create __init__.py file inside. If you using git, add !migrations/__init__.py to the .gitignore file

Upvotes: 1

Benoit Blanchon
Benoit Blanchon

Reputation: 14511

In my case, migrations didn't run because I used unittest.TestCase instead of django.test.TestCase.

Solution: replace

from unittest import TestCase

with

from django.test import TestCase

Upvotes: 0

jaandaldethu
jaandaldethu

Reputation: 63

I had a similar issue with manage.py test when I ran it in gitlab CI jobs, I was able to fix this by creating a variable called CI in my base.py settings file

base.py

"""
Set CI Variable to True when running: "manage.py test"
"""
CI = True

and in my settings.py:

if CI:
    class DisableMigrations(object):

        def __contains__(self, item):
            return True

        def __getitem__(self, item):
            return None # For Django 1.10+

MIGRATION_MODULES = DisableMigrations()

Upvotes: 1

liansheng
liansheng

Reputation: 308

python manage.py migrate auth

This works.

Upvotes: 4

valex
valex

Reputation: 5769

I've got same error beacuse one of my django applications have not migrations directory. Try to watch carefully all your apps folders which are present in INSTALLED_APPS.

Upvotes: 0

Diego Ponciano
Diego Ponciano

Reputation: 1463

Got the same issue, and since it happens on ./manage.py test, your migrations may be broken.
Since Django 1.7, there is a new setting called MIGRATION_MODULES, in which you configure your app's migration modules.
Adding the following workaround in settings.py (found here) skips migrations on tests, and solved it for me:

class DisableMigrations(object):

    def __contains__(self, item):
        return True

    def __getitem__(self, item):
        return "notmigrations"

MIGRATION_MODULES = DisableMigrations()

Upvotes: 11

Sebastian Wozny
Sebastian Wozny

Reputation: 17506

This should be solved by running:

python manage.py migrate auth
python manage.py migrate app
python manage.py migrate

Upvotes: 7

kunl
kunl

Reputation: 1195

You have not given any code so its difficult to understand your models relations. But can I guess that you have passed the ForeignKey model as an instance? Try to give it as a string

class MyModel(models.Model):
    fk_field = models.ForeignKey('path.to.other.model')    # same as from path.models import model

This usually works out. Hope it does for you too!

Upvotes: 2

Related Questions