Django ModelForm doesn't ingest derived form data correctly

Question

I'm having trouble getting a file upload/processing process to work, as part of a Wagtail CMS app I've been developing. I've only been working on the site on and off over the last six weeks but so far I've had quite a bit of success setting up a multilevel page model structure, new block types in streamfields, and ingesting images from an external cloud environment into the Wagtail Image store.

My aim is to have a model that lists a series of Paths (ie. lat/long sequences), the information for which is extracted from files that are uploaded via the Wagtail admin interface. The approach I've taken so far is to build it around Wagtail ModelAdmin so that I can maintain the path list (list, edit, delete) and override the add/create function so that the path files can be uploaded via drag-and-drop; the code for that has been derived from Wagtail's image and document admin apps.

Specifically, I'm finding the values of the fields in the model are not being set when using a model form. These values are derived from the contents of the uploaded file rather than directly from a POSTed form. Because the model values are not being set, saving the model and subsequent editing are affected.

Here are the simplified code snippets:

models.py:

from django.db import models

class AbstractPath(models.Model):
    name = models.CharField(max_length=100, blank=False, null=False)
    start_location = models.CharField(max_length=120, blank=False, null=False)
    start_timestamp = models.DateTimeField(blank=False, null=False)
    stop_location = models.CharField(max_length=120, blank=False, null=False)
    stop_timestamp = models.DateTimeField(blank=False, null=False)

    class Meta:
        abstract = True

    def __str__(self):
        return self.name

class RealPath(AbstractPath):
    average_speed = models.FloatField(blank=False, null=False)
    max_speed = models.FloatField(blank=False, null=False)

forms.py:

from wagtail.admin.forms import WagtailAdminModelForm
from wagtail.admin.edit_handlers import BaseFormEditHandler

class PathForm(WagtailAdminModelForm):
    permission_policy = paths_permission_policy

    class Meta:
        model = RealPath
        fields = '__all__'

class PathFormEditHandler(BaseFormEditHandler):
    base_form_class = PathForm

admin.py:

from wagtail.contrib.modeladmin.options import ModelAdmin, modeladmin_register
from wagtail.contrib.modeladmin.views import CreateView, EditView

from django.template.loader import render_to_string
from django.http import HttpResponseBadRequest, JsonResponse
from django.utils.encoding import force_str

class RealPathCreateView(CreateView):
    def get_template_names(self):
        return ['RealPath_create.html']

    def post(self, request, *args, **kwargs):
        if not request.is_ajax():
            return HttpResponseBadRequest("Cannot POST to this view without AJAX")

        if not request.FILES:
            return HttpResponseBadRequest("Must upload a file")

        # Get dict with data derived from the file - keys align with model fields
        path_form_data = pathfile_process(request.FILES['files[]'])

        # Build a form for validation
        RealPathForm = self.get_form_class()
        form = RealPathForm(path_form_data, { 'file': request.FILES['files[]'] })

        if form.is_valid():
            path = form.save(commit=False)

            # Temporary workaround to load path model with derived data - shouldn't be necessary?
            for f, v in path_form_data.items():
                setattr(path, f, v)

            path.save()

            return JsonResponse({
                'success': True,
                'path_id': int(path.id),
                'form': render_to_string('RealPath_edit.html', {
                    'path': path,
                    'form': RealPathForm(
                        instance=path, prefix='path-%d' % path.id
                    ),
                }, request=request),
            })
        else:
            # Validation error
            return JsonResponse({
                'success': False,
                'error_message': '
'.join(['
'.join([force_str(i) for i in v]) for k, v in form.errors.items()]),
            })

    def get_edit_handler(self):
        edit_handler = self.model_admin.get_edit_handler(
            instance=self.get_instance(), request=self.request
        )
        return edit_handler.bind_to(model=self.model_admin.model)


class RealPathAdmin(ModelAdmin):
    model = RealPath
    menu_label = "Real Path"
    menu_icon = "arrow-right"
    menu_order = 320
    add_to_settings_menu = False
    exclude_from_explorer = False
    list_display = ("name", "start_timestamp", "start_location", "stop_timestamp", "stop_location")
    form_fields_exclude = ["start_timestamp", "stop_timestamp", "average_speed"]

    create_view_class = RealPathCreateView

    def get_edit_handler(self, instance, request):
        return PathFormEditHandler(())


modeladmin_register(RealPathAdmin)

The path files are uploading fine, and are made available in the RealPathCreateView.post() method. The file is processed to extract the relevant data, and put into path_form_data. My expectation is that when the form is created with RealPathForm, it will create an instance of the RealPath model and populate the fields therein with that data. What I have found is that the data values are not populated; if at this point I attempt a save with form.save(commit=True), an exception is raised django.db.utils.IntegrityError: NOT NULL constraint failed: paths_realpath.start_timestamp.

Deeper investigation of the problem shows the form object has an empty fields attribute, which means the RealPath model fields were never being set, the form is then "validated" and the save fails because most of the data passed to the database layer were either None or 0.0. The full list of fields is being generated in django's ModelFormMetaclass.__new__() method, based on the contents of the model, but never passed down to the model itself.

I eventually implemented the workaround to set the model fields manually (as in the code above), but then I found the subsequent form rendering was also broken because of the need to iterate over the form fields - which weren't populated either. Clearly I should fix the first problem as it will fix the second (and probably others), but I cannot see where in the django code this transfer occurs and hence what changes I need to make to my code.

Thanks for your assistance.

Note: You will note that the RealPath model is based on an AbstractPath model - the reason for this is that I will have several types of real Paths with data derived from different sources. I've left that structure there in case it is the reason for the problem I'm experiencing.

Django ModelForm doesn't ingest derived form data correctly

Answers (1)

Related Questions

Django ModelForm doesn&#39;t ingest derived form data correctly

Answers (1)

Related Questions

Django ModelForm doesn't ingest derived form data correctly