CODEWITHSUNDEEP
django
Lem0n
Lem0n

Reputation: 1297

django fixtures (from dumpdata) failing when testing

I'm trying to dump data from my production server to use as test on my dev server, but I'm getting errors when running "./manage.py test" on the dev server specifing the fixture file created on the prod server.

Here are the tries I made based on google/stackoverflow search:

# python manage.py dumpdata --indent=4 --natural
error when running tests: IntegrityError: (1062, "Duplicate entry 'cms-agencies' for key 'app_label'")

# python manage.py dumpdata --exclude contenttypes --indent=4
error when running tests: IntegrityError: (1452, 'Cannot add or update a child row: a foreign key constraint fails (`test_current`.`django_admin_log`, CONSTRAINT `content_type_id_refs_id_288599e6` FOREIGN KEY (`content_type_id`) REFERENCES `django_content_type` (`id`))')

# python manage.py dumpdata --exclude contenttypes --natural --indent=4
error when running tests: IntegrityError: (1062, "Duplicate entry '14-add_agencies' for key 'content_type_id'")

# python manage.py dumpdata --exclude contenttypes --exclude auth --indent=4
error when running tests: IntegrityError: (1452, 'Cannot add or update a child row: a foreign key constraint fails (`test_current`.`django_admin_log`, CONSTRAINT `user_id_refs_id_c8665aa` FOREIGN KEY (`user_id`) REFERENCES `auth_user` (`id`))')

# python manage.py dumpdata --exclude contenttypes --exclude auth --natural --indent=4
error when running tests: IntegrityError: (1452, 'Cannot add or update a child row: a foreign key constraint fails (`test_current_abril`.`django_admin_log`, CONSTRAINT `user_id_refs_id_c8665aa` FOREIGN KEY (`user_id`) REFERENCES `auth_user` (`id`))')

I also tried removing "'init_command': 'SET storage_engine=INNODB'" from settings.py and still got 1062 errors.

I don't understand this problem. Shouldn't django just recreate the DB exactly as it was on the prod server when I load the fixtures?

Upvotes: 8

Views: 8750

Answers (4)

Mr. Napik
Mr. Napik

Reputation: 5677

The problem is, that if you use natural keys (natural-foreign keys in later versions of Django), Django will actually store many-to-many relationships inside parent objects. That is something you want. But then you cannot simply dump all tables, you must not include the many-to-many tables/models in your dump as you would then load same data twice - and boom, duplicates and IntegrityErrors.

For example, you should dump auth.User and auth.Group, but not auth.User_Groups. Look at example of a dump from Django 1.7:

{
    "model": "auth.group",
    "fields": {
        "permissions": [],
        "name": "ST"
    },
},
{
    "model": "auth.group",
    "fields": {
        "permissions": [],
        "name": "property_manager"
    },
},

{
    "model": "auth.user",
    "fields": {
        "username": "boss",
        "groups": [
            ["property_manager"],["ST"],
        ],
        "user_permissions": [],
    },
},

Following line creates a comprehensive dump of users/permissions and content types you can move to dev to get identical copy, including identical row order and primary keys (tested on Django 1.7):

python manage.py dumpdata auth.User auth.Group contenttypes auth.Permission --indent 4 --natural-foreign > users.json

Upvotes: 1

Nick
Nick

Reputation: 1258

your problem might be that you dumped out a unicode file but django expects a ascii file format when loading. This will happen using PowerShell on Windows if that applies to you.

Upvotes: 0

Michael Bylstra
Michael Bylstra

Reputation: 5222

I had similar problems and these args worked for me:

python manage.py dumpdata --natural --exclude auth.permission --exclude contenttypes --indent 4

I also had a lot of issues with the post_save signal creating objects twice. There's a fix for that: How do I prevent fixtures from conflicting with django post_save signal code?

Upvotes: 14

dm03514
dm03514

Reputation: 55952

I believe the errors are telling you exactly what is happening. Is app_label unique? I would guess it is. I would think you have two objects with the same app_label key value. (cms-agencies)

Additionally when you have a foreign key relationship you need to have the object that corresponds to the foreign key. Dump data doesn't do this because it only dumps one model.

Something like https://github.com/davedash/django-fixture-magic is great for this. It dumps your model AND all fk dependencies.

Upvotes: 1

Related Questions