How to optimize an image (file upload) in Django before it is added to the storage location?

Question

We are updating our backend storage for our Django project from a local disk store to an Amazon S3 bucket. Currently, we add the image, then optimize it and at a later time rsync it to our CDN. We control these steps so I just optimize after the upload and before the rsync.

We are moving to Amazon S3 and I would like to now optimize the images before they are uploaded to the S3 bucket, primarily so we don't upload to S3, then download in order to optimize and finally, re-upload. Why have three trips when we can probably do this in one.

My question is this: How can we intercept the upload to optimize the file before it's pushed to the storage backend, in this case, Amazon S3.

If it helps I am using amazon's boto library and django-storages-redux.

---

Answer 1

I had this question in draft form and realized I had never posted it. I did not find the solution on stack overflow so I thought I would add it as a Q&A post.

The solution is to override Django's TemporaryFileUploadHandler class. I also set the file size for uploads to zero so they all happen on disk and no in memory, though that might not be necessary.

      # encoding: utf-8
      from image_diet import squeeze
      import shutil
      import uuid

      from django.core.files import File
      from django.core.files.uploadhandler import TemporaryFileUploadHandler


      class CompressImageUploadHandler(TemporaryFileUploadHandler):
      """
      Run image squeeze on our temporary file before upload to S3
      """

        def __init__(self, *args, **kwargs):
            self.image_types = ('image/jpeg', 'image/png')
            self.file_limit = 200000
            self.overlay_fields = (
                'attribute_name',  
            )
            self.skip_compress_fields = (
                'attribute_name',  
            )
            super(CompressImageUploadHandler, self).__init__(*args, **kwargs)

        def compress_image(self):
            """
            For image files we need to compress them, but we need to do some
            trickery along the way. We need to close the file, pass it to
            image_diet.squeeze, then reopen the file with the same file name
            """

            # if it's an image and small enough. Squeeze.
            if (self.file.size < self.file_limit and
                    self.field_name not in self.skip_compress_fields):
                # the beginning is a good place to start.
                self.file.seek(0)
                # let's squeeze this image. 
                # first, make a copy.
                file_name = self.file.name
                file_content_type = self.file.content_type
                copy_path = u"{}{}".format(
                    self.file.temporary_file_path(),
                    str(uuid.uuid4())[:8]
                )
                shutil.copyfile(
                    self.file.temporary_file_path(),
                    copy_path
                )
                # closed please. image_squeeze updates on an open file
                self.file.close()
                squeeze(copy_path)
                squeezed_file = open(copy_path)
                self.file = File(squeezed_file)
                # now reset some of the original values
                self.file.name = file_name
                self.file.content_type = file_content_type

        def screenshot_overlay(self):
            """
            Apply the guarantee_image_overlay method on screenshots
            """
            if self.field_name in self.overlay_fields:
                # this is a custom method that adds an overlay to the upload image if it's in the tuple of overlay_fields
                guarantee_image_overlay(self.file.temporary_file_path())
                # we have manipulated file, back to zero
                self.file.seek(0)

        def file_complete(self, file_size):
            """
            Return the file object, just run image_squeeze against it.
            This happens before the file object is uploaded to Amazon S3.
            While the pre_save hook happens after the Amazon upload.
            """
            self.file.seek(0)
            self.file.size = file_size

            if self.content_type in self.image_types:
                # see if we apply the screenshot overlay.
                self.screenshot_overlay()
                self.compress_image()

            return super(CompressImageUploadHandler, self).file_complete(file_size)