Reputation: 471
I need to dump and load fixtures of model objects without usage of primary keys. The model is flat. I know about natural keys in Django, spent a lot of time reading the documentation, but all documentation has solutions only for usage natural keys instead of relations (fk / m2m). This is totally not what I need.
I need something like this:
(models.py)
class Template(models.Model):
name = models.CharField(_('name'), max_length=100)
content = models.TextField(_('content'), blank=True)
def natural_key(self):
return (self.name,)
(fixture1.json)
[
{
"pk": null,
"model": "dbtemplates.Template",
"fields" {
"content": "",
"name": "product"
}
}
]
And after command
./manage.py <SOME_LOADDATA_COMMAND> fixture1.json --natural
I need to update my Template object which has name "product" or insert it.
The standard Django commands don't do this. Please, help me with any solution. Maybe there are some libraries for this? I'm confusing.
Django 1.6. Python 2.7
Upvotes: 2
Views: 1326
Reputation: 471
Based on the answer of régis-b I wrote some code that allows use natural keys in Django 1.6 "loaddata" management command without upgrade to 1.7. I chose this way because a full upgrading of my project may be painful. This solution can be considered as a temporary.
tree structure:
├── project_main_app
│ ├── __init__.py
│ ├── backports
│ │ ├── __init__.py
│ │ └── django
│ │ ├── __init__.py
│ │ └── deserializer.py
│ └── monkey.py
project_main_app/backports/django/deserializer.py
from __future__ import unicode_literals
from django.conf import settings
from django.core.serializers import base
from django.core.serializers.python import _get_model
from django.db import models, DEFAULT_DB_ALIAS
from django.utils.encoding import smart_text
from django.utils import six
def Deserializer(object_list, **options):
"""
Deserialize simple Python objects back into Django ORM instances.
It's expected that you pass the Python objects themselves (instead of a
stream or a string) to the constructor
"""
db = options.pop('using', DEFAULT_DB_ALIAS)
ignore = options.pop('ignorenonexistent', False)
models.get_apps()
for d in object_list:
# Look up the model and starting build a dict of data for it.
Model = _get_model(d["model"])
data = {Model._meta.pk.attname: Model._meta.pk.to_python(d.get("pk", None))}
m2m_data = {}
model_fields = Model._meta.get_all_field_names()
# Handle each field
for (field_name, field_value) in six.iteritems(d["fields"]):
if ignore and field_name not in model_fields:
# skip fields no longer on model
continue
if isinstance(field_value, str):
field_value = smart_text(field_value, options.get("encoding", settings.DEFAULT_CHARSET), strings_only=True)
field = Model._meta.get_field(field_name)
# Handle M2M relations
if field.rel and isinstance(field.rel, models.ManyToManyRel):
if hasattr(field.rel.to._default_manager, 'get_by_natural_key'):
def m2m_convert(value):
if hasattr(value, '__iter__') and not isinstance(value, six.text_type):
return field.rel.to._default_manager.db_manager(db).get_by_natural_key(*value).pk
else:
return smart_text(field.rel.to._meta.pk.to_python(value))
else:
m2m_convert = lambda v: smart_text(field.rel.to._meta.pk.to_python(v))
m2m_data[field.name] = [m2m_convert(pk) for pk in field_value]
# Handle FK fields
elif field.rel and isinstance(field.rel, models.ManyToOneRel):
if field_value is not None:
if hasattr(field.rel.to._default_manager, 'get_by_natural_key'):
if hasattr(field_value, '__iter__') and not isinstance(field_value, six.text_type):
obj = field.rel.to._default_manager.db_manager(db).get_by_natural_key(*field_value)
value = getattr(obj, field.rel.field_name)
# If this is a natural foreign key to an object that
# has a FK/O2O as the foreign key, use the FK value
if field.rel.to._meta.pk.rel:
value = value.pk
else:
value = field.rel.to._meta.get_field(field.rel.field_name).to_python(field_value)
data[field.attname] = value
else:
data[field.attname] = field.rel.to._meta.get_field(field.rel.field_name).to_python(field_value)
else:
data[field.attname] = None
# Handle all other fields
else:
data[field.name] = field.to_python(field_value)
# The key block taken from Django 1.7 sources
obj = build_instance(Model, data, db)
yield base.DeserializedObject(obj, m2m_data)
# This is also taken from Django 1.7 sources
def build_instance(Model, data, db):
"""
Build a model instance.
If the model instance doesn't have a primary key and the model supports
natural keys, try to retrieve it from the database.
"""
obj = Model(**data)
if (obj.pk is None and hasattr(Model, 'natural_key') and
hasattr(Model._default_manager, 'get_by_natural_key')):
natural_key = obj.natural_key()
try:
obj.pk = Model._default_manager.db_manager(db).get_by_natural_key(*natural_key).pk
except Model.DoesNotExist:
pass
return obj
project_main_app/monkey.py
def patch_all():
import django.core.serializers.python
import project_main_app.backports.django.deserializer
# Patch the Deserializer
django.core.serializers.python.Deserializer = project_main_app.backports.django.deserializer.Deserializer
project_main_app/init.py
from project_main_app.monkey import patch_all
patch_all()
So after this I just add some things and my model becomes like
class TemplateManager(models.Manager):
"""1"""
def get_by_natural_key(self, name):
return self.get(name=name)
class Template(models.Model):
name = models.CharField(_('name'), max_length=100)
content = models.TextField(_('content'), blank=True)
objects = TemplateManager() # 2
def natural_key(self):
"""3"""
return (self.name,)
and if fixtures has an empty pk like
[
{
"pk": null,
"model": "dbtemplates.Template",
"fields": {
"content": "Some content",
"name": "product"
}
}
]
the standard command ./manage.py loaddata dbtemplates.Template updates or inserts object matching by name field.
Warning: all of natural key components (like a "name" in my case) must have unique values in the database. The proper way is to set them unique by adding the argument "unique=True" when defining a model.
Upvotes: 0
Reputation: 10618
Django 1.6 does not provide a way to dump data with natural primary keys, but Django 1.7 does.
--natural-primary
option): https://docs.djangoproject.com/en/1.7/ref/django-admin/#dumpdata-app-label-app-label-app-label-modelUnfortunately, the use_natural_primary_keys
keyword argument is not supported by the base Django 1.6 serializer: https://github.com/django/django/blob/1.6.11/django/core/serializers/base.py#L20
So I suggest you either upgrade to django 1.7 (which I totally understand is not always possible) or you write your own serializer, drawing inspiration from the base Django 1.7 serializer (https://github.com/django/django/blob/1.7.11/django/core/serializers/base.py).
Upvotes: 3