Add a set of 'sub-forms' (probably form set) to a ModelForm

Question

I'm making some progress in solving this. What I want to model is this:

I have a model for materials, a model for projects, and an intermediary model to specify more data to the many-to-many relationship between them. The objective is to allow a project not only to have multiple materials but also to specify a quantity for each.

These are my models:

class Material(models.Model):
    id = models.AutoField(primary_key=True)
    name = models.CharField(max_length=200)
    description = models.CharField(max_length=3000, blank=True)
    unit = models.CharField(max_length=20)
    price = models.DecimalField(max_digits=9, decimal_places=2, validators=[MinValueValidator(0)])
    created_by = models.ForeignKey(to=User, on_delete=models.CASCADE, related_name='materials')
    created_on = models.DateTimeField(auto_now_add=True)

    def __str__(self):
        return f'{self.name} (${self.price}/{self.unit})'


class Project(models.Model):
    id = models.AutoField(primary_key=True)
    name = models.CharField(max_length=200)
    description = models.CharField(max_length=3000, blank=True)
    materials = models.ManyToManyField(to=Material, through='ProjectMaterialSet', related_name='projects')
    created_by = models.ForeignKey(to=User, on_delete=models.CASCADE)
    created_on = models.DateTimeField(auto_now_add=True)

    def __str__(self):
        return f'{self.name}'

class ProjectMaterialSet(models.Model):
    id = models.AutoField(primary_key=True)
    project = models.ForeignKey(to=Project, on_delete=models.CASCADE)
    material = models.ForeignKey(to=Material, on_delete=models.CASCADE)
    material_qty = models.PositiveIntegerField(default=1)

Now, I have successfully rendered the project form containing a checkbox list of the materials (without the quantity feature), another form listing just one material as a checkbox, and a field for the quantity (a form of the ProjectMaterialSet model).

Now, the part I'm struggling with is how to combine both. So far, I think what I need to do is have a query set of the available materials in the ProjectForm, to then add to it a series of sub-forms (here is where I think I should use formsets, however, I'm unsure how) that passes each material pk so I can render each form as a ModelMultipleChoiceField so I get to use the checkbox widget for each material.

My natural instinct as someone not related with formsets is to for-loop over the materials and create form instances that save to a dict or something around those lines. However, I read on the Django forum a good phrase about using Django features instead of hacking a solution, and I do feel the formsets can be used for this, I'm just unsure how.

Clarification: I need to pass each material pk to the ProjectMaterialSetForm so I can get the checkbox for each material (as I'm getting a query set with just one result since I'm filtering by pk), and link a quantity input to that single checkbox.

I feel I have most of it done, I'm just not sure how to add the sub-forms, and if someone could also help about the custom save and validation required, that'd be amazing!

Here are my forms:

class ProjectForm(forms.ModelForm):

    class Meta:
        model = Project
        fields = ('name', 'description')

        widgets = {
            'name': forms.TextInput(attrs={'class': 'form-control', 'placeholder': 'Enter the project name'}),
            'description': forms.Textarea(attrs={'class': 'form-control', 'placeholder': 'Enter the project description', 'rows': '5'})
        }

    def __init__(self, *args, **kwargs):

        # Store the user object and materials QuerySet
        self.user = kwargs.pop('user') if 'user' in kwargs else None
        self.materials = Material.objects.filter(created_by=self.user) if self.user is not None else None # This is a QuerySet

        # Call parent __init__ method, which we are modifying
        super(ProjectForm, self).__init__(*args, **kwargs)

        # Somehow create a set of forms, one for each material
            # Perhaps using formsets is the ideal solution, however, I keep finding
            # myself going to the route of a dict with forms and adding them as I
            # iterate over the materials query set. 

            # Since formsets are a thing, I'm pretty sure I can somehow create a batch
            # of forms (probably the extra argument is the key here), to then somehow
            # add the material_id to each of them.


class ProjectMaterialSetForm(forms.ModelForm):

    class Meta:
        model = ProjectMaterialSet
        fields = ('material', 'material_qty')

        material = forms.ModelMultipleChoiceField(queryset=None)
        
        widgets = {
            'material': forms.CheckboxSelectMultiple(attrs={'class':'form-check-input'}),
            'material_qty': forms.NumberInput(attrs={'class':'form-control'})
        }

    def __init__(self, *args, **kwargs):
        material_id = kwargs.pop('material_id') if 'material_id' in kwargs else None
        super(ProjectMaterialSetForm, self).__init__(*args, **kwargs)
        self.fields['materials'].queryset = Material.objects.filter(pk=material_id)

Answer 1

I have come into a similar situation, and I ended up with customizing model formset.

class ProjectMaterialSetFormSet(forms.BaseModelFormSet):
    # some required / optional attributes
    extra = 0
    can_delete = True
    can_order = False
    max_num = 1000
    validate_max = False
    # If you want to enforce each project to have at least
    # one material, set validate_min to True
    min_num = 1
    validate_min = False
    absolute_max = 1000
    can_delete_extra = True
    renderer = forms.renderers.get_default_renderer()

    model = ProjectMaterialSet
    class ProjectMaterialSetForm(forms.ModelForm):
        # Add an extra name field to help form rendering
        material_name = forms.CharField(max_length=200, required=False)
        class Meta:
            model = ProjectMaterialSet
            fields = ['material', 'material_qty']
    form = ProjectMaterialSetForm

    # ProjectForm is like another management form for the formset: 
    # rendered, validated and saved together with formset. 
    class ProjectForm(forms.ModelForm):
        class Meta:
            model = Project
            fields = ['name', 'description']

    @cached_property
    def project_form(self):
        if self.is_bound:
            form = self.ProjectForm(self.data, self.files, prefix=self.prefix)
            form.full_clean()
        else:
            form = self.ProjectForm(prefix=self.prefix)
        return form

    def clean(self):
        super().clean()
        if not self.project_form.is_valid():
            raise forms.ValidationError('Project form invalid')

    def save(self, commit=True):
        project = self.project_form.save(commit=commit)
        for form in self:
            form.instance.project = project
        return super(ProjectMaterialSetFormSet, self).save(commit=commit)
        # return project if you want the instance for further operation,
        # but just don't forget to call super().save()

    def get_queryset(self):
        # Override get_queryset to return none if not specified. 
        # Otherwise, it just returns all. 
        if not hasattr(self, '_queryset'):
            if self.queryset is not None:
                qs = self.queryset
            else:
                qs = self.model.objects.none()
            if not qs.ordered:
                qs = qs.order_by(self.model._meta.pk.name)
            self._queryset = qs
        return self._queryset

When rendering the formset, provide all materials as initial data

def project_create_view(request):
    if request.method == 'POST':
        formset = ProjectMaterialSetFormSet(request.POST, prefix='proj-mats')
        if formset.is_valid():
            proj_mat_set = formset.save()
    else:
        initial = Material.objects.values(material=F('id'), material_name=F('name')).annotate(DELETE=Value(True))
        # Don’t forget to set prefix same as above. 
        #formset = ProjectMaterialSetFormSet(initial=initial)
        formset = ProjectMaterialSetFormSet(initial=initial, prefix='proj-mats')
        # Not so sure about why minus 1 here. I did it 
        # in my code. Remove it if you find problem. 
        formset.extra = len(initial) - 1
    context = {'formset': formset}
    return render(request, 'path/to/template.html', context)

When rendering the template, don't forget to render project_form. If you tend to render each child form separately, which is recommended here, also don't forget to render management form.

{{ formset.project_form }}
{{ formset.management_form }}
{% for form in formset %}
<input type="checkbox" name="{{ form.prefix }}-DELETE" {{ form.DELETE.value|yesno:',checked' }} id="id-{{ form.prefix }}-DELETE">
<input type="hidden" name="{{ form.prefix }}-material" value="{{ form.material.value }}">
<input type="hidden" name="{{ form.prefix }}-material_name" value="{{ form.material_name.value }}">
<label for="id-{{ form.prefix }}-DELETE">{{ form.material_name.value }}</label>
<input type="text" name="{{ form.prefix }}-material_qty" value="{{ form.material_qty.value }}" pattern="\d+">
{% endfor %}

--- Update ---

The idea comes from ManagementForm of Django's Formset, which contains two meta fields: {prefix}-TOTAL_FORMS and {prefix}-INITIAL_FORMS.

This looks very similar in your case: a formset for ProjectMaterialSet, and a form for Project that tightly related to the formset.

Then what to do next is to mimic how Django handles management_form, which can be found in source code:

The management_form, as a @cached_property method to formset, will create a ManagementForm instance when accessed. This form will get validated when you call .is_valid on formset, which accesses formset.errors, which then triggers formset.full_clean, which then triggers management_form.is_valid.

In your case, the clean route can be simplified by putting the project_form.is_valid into formset.clean.

Next question is how to create such formset class, and instantiate an instance in the view function.

Django has a shortcut modelformset_factory to help us create ModelFormSet. If you look into its source code, it is nothing more of defining an object that inherits from BaseModelFormSet class, with some controlling attributes (extra, can_delete, can_order, etc).

The you can totally define a child ModelFormSet class, setup all those attributes as if you were calling modelformset_factory, override clean method to include validation on ProjectForm, and override save method to save the project first, and then to save ProjectMaterialSet with project attribute set to the newly created project