Limit Amount Of Files A User Can Upload

Question

I have a multi-file upload and want to limit users to 3 uploads each. My problem is that I need to know how many files a user has already created in the DB and how many they are currently uploading (they can upload multiple files at once, and can upload multiple times).

I have attempted many things, including:

Creating a validator (the validator was passed the actual file being added, not a model, so I couldn't access the model to get it's id to call if StudentUploadedFile.objects.filter(student_lesson_data=data.id).count() >= 4:).

Doing the validation in clean(self): (clean was only passed one instance at a time and the DB isn't updated till all files are cleaned, so I could count the files already in the DB but couldn't count how many were currently being uploaded).

Using a pre-save method (If the DB was updated between each file being passed to my pre-save method it would work, but the DB is only updated after all the files being uploaded have passed through my pre-save method).

My post-save attempt:

@receiver(pre_save, sender=StudentUploadedFile)
def upload_file_pre_save(sender, instance, **kwargs):

    if StudentUploadedFile.objects.filter(student_lesson_data=instance.data.id).count() >= 4:
        raise ValidationError('Sorry, you cannot upload more than three files')

edit:

models.py

class StudentUploadedFile(models.Model):
    student_lesson_data = models.ForeignKey(StudentLessonData, related_name='student_uploaded_file', on_delete=models.CASCADE)
    student_file = models.FileField(upload_to='module_student_files/', default=None)

views.py

class StudentUploadView(View):
    def get(self, request):
        files_list = StudentUploadedFile.objects.all()
        return render(self.request, 'users/modules.html', {'student_files': files_list})

    def post(self, request, *args, **kwargs):
        form = StudentUploadedFileForm(self.request.POST, self.request.FILES)
        form.instance.student_lesson_data_id = self.request.POST['student_lesson_data_id']

        if form.is_valid():
            uploaded_file = form.save()

            # pass uploaded_file data and username so new file can be added to students file list using ajax
            # lesson_id is used to output newly added file to corresponding newly_added_files div
            data = {'is_valid': True, 'username': request.user.username, 'file_id': uploaded_file.id, 'file_name': uploaded_file.filename(),
            'lesson_id': uploaded_file.student_lesson_data_id, 'file_path': str(uploaded_file.student_file)}
        else:
            data = {'is_valid': False}
        return JsonResponse(data)

template.py

<form id='student_uploaded_file{{ item.instance.id }}'>
                                                {% csrf_token %}
                                                <a href="{% url 'download_student_uploaded_file' username=request.user.username file_path=item.instance.student_file %}" target='_blank'>{{ item.instance.filename }}</a>
                                                <a href="{% url 'delete_student_uploaded_file' username=request.user.username file_id=item.instance.id %}" class='delete' id='{{ item.instance.id }}'>Delete</a>
                                            </form>

js

$(function () {
    // open file explorer window
    $(".js-upload-photos").on('click', function(){
        // concatenates the id from the button pressed onto the end of fileupload class to call correct input element
        $("#fileupload" + this.id).click();
     });

    $('.fileupload_input').each(function() {
        $(this).fileupload({
            dataType: 'json',
            done: function(e, data) { // process response from server
            // add newly added files to students uploaded files list
            if (data.result.is_valid) {
                $("#newly_added_files" + data.result.lesson_id).prepend("<form id='student_uploaded_file" + data.result.file_id +
                "'><a href='/student_hub/" + data.result.username + "/download_student_uploaded_file/" +
                data.result.file_path + "' target='_blank'>" + data.result.file_name + "</a><a href='/student_hub/" + data.result.username +
                "/delete_student_uploaded_file/" + data.result.file_id + "/'  class='delete' id=" + data.result.file_id + ">Delete</a></form>")
            }
            }
        });
    });

UPDATE: forms.py

class StudentUploadedFileForm(forms.ModelForm):
    student_file = forms.FileField(widget=forms.ClearableFileInput(attrs={'multiple': True}))

view.py

class StudentUploadView(View):
    model = StudentUploadedFile
    max_files_per_lesson = 3

    def post(self, request, *args, **kwargs):
        lesson_data_id = request.POST['student_lesson_data_id']
        current_files_count = self.model.objects.filter(
            student_lesson_data_id=lesson_data_id
        ).count()
        avail = self.max_files_per_lesson - current_files_count
        file_list = request.FILES.getlist('student_file')
        print(len(file_list))
        if avail - len(file_list) < 0:
            return JsonResponse(data={
                'is_valid': False,
                'reason': f'Too many files: you can only upload {avail}.'
            })
        else:
            for f in file_list:
                print(f)
                
        data = {'test': True}
        return JsonResponse(data)

Thank you.

Answer 1

You can use the below setting to set the number of files that can be uploaded. (New in Django 3.2.18. )

DATA_UPLOAD_MAX_NUMBER_FILES

The maximum number of files that may be received via POST in a multipart/form-data encoded request before a SuspiciousOperation (TooManyFiles) is raised. You can set this to None to disable the check. Applications that are expected to receive an unusually large number of file fields should tune this setting.

The number of accepted files is correlated to the amount of time and memory needed to process the request. Large requests could be used as a denial-of-service attack vector if left unchecked. Since web servers don’t typically perform deep request inspection, it’s not possible to perform a similar check at that level.

The default value is 100.

Answer 2

I guess that you can use multi file upload in Django hasn't trickled to the community yet. Excerpt:

If you want to upload multiple files using one form field, set the multiple HTML attribute of field’s widget:

# forms.py

from django import forms

class FileFieldForm(forms.Form):
    file_field = forms.FileField(widget=forms.ClearableFileInput(attrs={'multiple': True}))

Your form and view structure is also very contrived, excluding fields from a form and then setting values injected via HTML form on the model instance. However, with the code shown, the model instance would never exist as the form has no pk field. Anyway - to focus on the problem that needs fixing...

In the form, request.FILES is now an array:

class StudentUploadView(View):
    model = StudentUploadedFile
    max_files_per_lesson = 3
    def post(request, *args, **kwargs):
        lesson_data_id = request.POST['student_lesson_data_id']
        current_files_count = self.model.objects.filter(
            student_lesson_data_id=lesson_data_id
        ).count()
        avail = self.max_files_per_lesson - current_files_count
        file_list = request.FILES.get_list('student_file')
        if avail - len(file_list) < 0:
            return JsonResponse(data={
                'is_valid': False,
                'reason': f'Too many files: you can only upload {avail}.'
            })
        else:
            # create one new instance of self.model for each file
            ...

Addressing comments: From an aesthetic perspective, you can do a lot with styling...

However, uploading async (separate POST requests), complicates validation and user experience a lot:

• The first file can finish after the 2nd, so which are you going to deny if the count is >3.
• Frontend validation is hackable, so you can't rely on it, but backend validation is partitioned into several requests, which from the user's point of view is one action.
• But with files arriving out of order, some succeeding and some failing, how are you going to provide feedback to the user?
• If 1, 3 and 4 arrive, but user cares more about 1, 2, 3 - user has to take several actions to correct the situation.

One post request:

• There are no out of order arrivals
• You can use a "everything fails or everything succeeds" approach, which is transparent to the end user and easy to correct.
• It's likely that file array order is user preferred order, so even if you allow partial success, you're likely to do the right thing.

Answer 3

I've tried using a PyPi package and it works flawlessly. I'm gonna go out on a limb here and assume that you are open to editing the package code to fix any errors you encounter due to compatibility issues since most of the packages that haven't been updated in quite a while might face them.

To solve the problem of limiting the number of files a user can upload, django-multiuploader package would be of enormous help and would honestly do more than you ask for. And yes, it uses JQuery form for uploading multiple files.

How to use it?

Installation and pre-usage steps

Installation

pip install django-multiuploader
python3 manage.py syncdb
python3 manage.py migrate multiuploader

In your settings.py file :

MULTIUPLOADER_FILES_FOLDER = ‘multiuploader’ # - media location where to store files

MULTIUPLOADER_FILE_EXPIRATION_TIME = 3600  # - time, when the file is expired (and it can be cleaned with clean_files command).

MULTIUPLOADER_FORMS_SETTINGS =

{
'default': {
    'FILE_TYPES' : ["txt","zip","jpg","jpeg","flv","png"],
    'CONTENT_TYPES' : [
            'image/jpeg',
            'image/png',
            'application/msword',
            'application/vnd.openxmlformats-officedocument.wordprocessingml.document',
            'application/vnd.ms-excel',
            'application/vnd.openxmlformats-officedocument.spreadsheetml.sheet',
            'application/vnd.ms-powerpoint',
            'application/vnd.openxmlformats-officedocument.presentationml.presentation',
            'application/vnd.oasis.opendocument.text',
            'application/vnd.oasis.opendocument.spreadsheet',
            'application/vnd.oasis.opendocument.presentation',
            'text/plain',
            'text/rtf',
                ],
    'MAX_FILE_SIZE': 10485760,
    'MAX_FILE_NUMBER':5,
    'AUTO_UPLOAD': True,
},
'images':{
    'FILE_TYPES' : ['jpg', 'jpeg', 'png', 'gif', 'svg', 'bmp', 'tiff', 'ico' ],
    'CONTENT_TYPES' : [
        'image/gif',
        'image/jpeg',
        'image/pjpeg',
        'image/png',
        'image/svg+xml',
        'image/tiff',
        'image/vnd.microsoft.icon',
        'image/vnd.wap.wbmp',
        ],
    'MAX_FILE_SIZE': 10485760,
    'MAX_FILE_NUMBER':5,
    'AUTO_UPLOAD': True,
},
'video':{
    'FILE_TYPES' : ['flv', 'mpg', 'mpeg', 'mp4' ,'avi', 'mkv', 'ogg', 'wmv', 'mov', 'webm' ],
    'CONTENT_TYPES' : [
        'video/mpeg',
        'video/mp4',
        'video/ogg',
        'video/quicktime',
        'video/webm',
        'video/x-ms-wmv',
        'video/x-flv',
        ],
    'MAX_FILE_SIZE': 10485760,
    'MAX_FILE_NUMBER':5,
    'AUTO_UPLOAD': True,
},
'audio':{
    'FILE_TYPES' : ['mp3', 'mp4', 'ogg', 'wma', 'wax', 'wav', 'webm' ],
    'CONTENT_TYPES' : [
        'audio/basic',
        'audio/L24',
        'audio/mp4',
        'audio/mpeg',
        'audio/ogg',
        'audio/vorbis',
        'audio/x-ms-wma',
        'audio/x-ms-wax',
        'audio/vnd.rn-realaudio',
        'audio/vnd.wave',
        'audio/webm'
        ],
    'MAX_FILE_SIZE': 10485760,
    'MAX_FILE_NUMBER':5,
    'AUTO_UPLOAD': True,
}}

Take note of that MAX_FILE_NUMBER, right within their lies the answer to your question. Have a look at the source once you install this and try implementing it on your own if you want. It might be fun.

Refer for further instructions : django-multiuploader package on pypi

Answer 4

So the gist of it is that you are using jQuery .each to upload images via AJAX. Each POST request to your Django view is a single file upload, but there might be multiple requests at the same time.

Try this:

forms.py:

class StudentUploadedFileForm(forms.ModelForm):

    class Meta:
        model = StudentUploadedFile
        fields = ('student_file', )

    def __init__(self, *args, **kwargs):
        """Accept a 'student_lesson_data' parameter."""
        self._student_lesson_data = kwargs.pop('student_lesson_data', None)
        super(StudentUploadedFileForm, self).__init__(*args, **kwargs)

    def clean(self):
        """
        Ensure that the total number of student_uploaded_file instances that
        are linked to the student_lesson_data parameter are within limits."""
        cleaned_data = super().clean()
        filecount = self._student_lesson_data.student_uploaded_file.count()
        if filecount >= 3:
            raise forms.ValidationError("Sorry, you cannot upload more than three files")
        return cleaned_data

views.py:

class StudentUploadView(View):
    def get(self, request):
        # stuff ...

    def post(self, request, *args, **kwargs):
        sld_id = request.POST.get('student_lesson_data_id', None)
        student_lesson_data = StudentLessonData.objects.get(id=sld_id)
        form = StudentUploadedFileForm(
            request.POST,
            request.FILES,
            student_lesson_data=student_lesson_data
        )

        if form.is_valid():
            uploaded_file = form.save()
            # other stuff ...