Getting two "https://" in the url of image

Question

I'm developing a project using Django and Django Rest Framework in which I have to save an image in a model. I'm using S3 bucket as a storage device. I'm able to upload the image and save it to the model.

The Problem

When getting response (either getting a single object or an array of objects), I'm getting the url of image with two https://. This only happens when I'm using the Django server that's hosted on AWS Ec2. The url of the image is returned normally when using localhost, the problem is with static files as well (but they aren't used, only the admin panel and rest-framework templates use it)

Example:

When calling the API from the hosted server

This is the response. Notice the image field.

[
    {
        "id": 5,
        "image": "https://https://d2to87w45k79nd.cloudfront.net/media/testimonies/Myron/Section_2_img.png",
        "name": "Myron",
        "message": "Cool Website",
        "position": "CEO",
        "company": "ME Ltd."
    },
    {
        "id": 6,
        "image": "https://https://d2to87w45k79nd.cloudfront.net/media/testimonies/phoenix0347/Section_2_img.png",
        "name": "phoenix0347",
        "message": "askjn",
        "position": "false",
        "company": "false"
    },
    {
        "id": 7,
        "image": "https://https://d2to87w45k79nd.cloudfront.net/media/testimonies/Kushagra%20Gupta/Section_9.png",
        "name": "Kushagra Gupta",
        "message": "jksabdsadb",
        "position": "1jb",
        "company": "sajd"
    },
    {
        "id": 8,
        "image": "https://https://d2to87w45k79nd.cloudfront.net/media/testimonies/jksadb/hero_img.png",
        "name": "jksadb",
        "message": "akjsbasjdb",
        "position": "213u3",
        "company": "129ujieo2"
    }
]

The same API when called from localhost gives

A response like this. Again notice the image field!

[
    {
        "id": 5,
        "image": "https://d2to87w45k79nd.cloudfront.net/media/testimonies/Myron/Section_2_img.png",
        "name": "Myron",
        "message": "Cool Website",
        "position": "CEO",
        "company": "ME Ltd."
    },
    {
        "id": 6,
        "image": "https://d2to87w45k79nd.cloudfront.net/media/testimonies/phoenix0347/Section_2_img.png",
        "name": "phoenix0347",
        "message": "askjn",
        "position": "false",
        "company": "false"
    },
    {
        "id": 7,
        "image": "https://d2to87w45k79nd.cloudfront.net/media/testimonies/Kushagra%20Gupta/Section_9.png",
        "name": "Kushagra Gupta",
        "message": "jksabdsadb",
        "position": "1jb",
        "company": "sajd"
    },
    {
        "id": 8,
        "image": "https://d2to87w45k79nd.cloudfront.net/media/testimonies/jksadb/hero_img.png",
        "name": "jksadb",
        "message": "akjsbasjdb",
        "position": "213u3",
        "company": "129ujieo2"
    }
]

I don't know what is causing the issue... I'm using boto3, django-storages python packages to achieve the storage in S3.

I'm providing my settings.py, storages.py, views.py, serializers.py

settings.py

# Static files (CSS, JavaScript, Images)
USE_S3 = not DEBUG

if USE_S3:  # In Production Use CDN(Amazon CloudFront in this case)
    # AWS Settings
    AWS_ACCESS_KEY_ID = os.getenv('AWS_ACCESS_ID')
    AWS_SECRET_ACCESS_KEY = os.getenv('AWS_SECRET_KEY')
    AWS_STORAGE_BUCKET_NAME = os.getenv('AWS_S3_BUCKET_NAME')
    AWS_DEFAULT_ACL = None
    AWS_CLOUDFRONT_DOMAIN = os.getenv('AWS_CLOUDFRONT_DOMAIN')
    AWS_S3_OBJECT_PARAMETERS = {'CacheControl': 'max-age=86400'}

    # s3 static settings
    STATIC_LOCATION = 'static'
    STATIC_URL = f'https://{AWS_CLOUDFRONT_DOMAIN}/{STATIC_LOCATION}/'
    STATICFILES_STORAGE = 'backend.storage_backends.StaticStorage'

    # media settings
    PUBLIC_MEDIA_LOCATION = 'media'
    MEDIA_URL = f'{AWS_CLOUDFRONT_DOMAIN}/{PUBLIC_MEDIA_LOCATION}/'
    DEFAULT_FILE_STORAGE = 'backend.storage_backends.MediaStorage'
else:  # In Development use local storage (I can also use s3 if I wish to in development by setting DEBUG to false)
    STATIC_URL = '/static/'
    STATIC_ROOT = os.path.join(BASE_DIR, 'static/')

    MEDIA_URL = '/media/'
    MEDIA_ROOT = os.path.join(BASE_DIR, 'media/')

storages.py

from django.conf import settings
from storages.backends.s3boto3 import S3Boto3Storage


class StaticStorage(S3Boto3Storage):
    location = settings.STATIC_LOCATION
    default_acl = settings.AWS_DEFAULT_ACL

    def __init__(self, **kwargs):
        kwargs['custom_domain'] = settings.AWS_CLOUDFRONT_DOMAIN
        super(StaticStorage, self).__init__(**kwargs)


class MediaStorage(S3Boto3Storage):
    location = settings.PUBLIC_MEDIA_LOCATION
    default_acl = settings.AWS_DEFAULT_ACL
    file_overwrite = False

    def __init__(self, **kwargs):
        kwargs['custom_domain'] = settings.AWS_CLOUDFRONT_DOMAIN
        super(MediaStorage, self).__init__(**kwargs)

serializer.py

from rest_framework import serializers

from .models import Testimonies


class TestimonySerializer(serializers.ModelSerializer):
    image = serializers.ImageField()

    @staticmethod
    def get_image(instance):
        try:
            print(instance.image.url)
            return instance.image.url
        except Exception as e:
            print(e)
            return ''

    class Meta:
        model = Testimonies
        fields = '__all__'

views.py

from rest_framework import status
from rest_framework.generics import GenericAPIView
from rest_framework.parsers import MultiPartParser, FormParser
from rest_framework.response import Response

from .serializers import TestimonySerializer
from .models import Testimonies


class Testimony(GenericAPIView):
    serializer_class = TestimonySerializer
    queryset = Testimonies.objects.all()
    lookup_field = 'id'
    parser_classes = [MultiPartParser, FormParser]

    def get(self, req, *args, **kwargs):
        # Get single Testimony
        if 'id' in kwargs:
            testimony = self.get_object()
            serializer = self.get_serializer(testimony)
            return Response(serializer.data, status=status.HTTP_200_OK)

        # Get All Testimonies
        testimonies = self.get_queryset()
        serializer = self.get_serializer(testimonies, many=True)
        return Response(serializer.data, status=status.HTTP_200_OK)

    def post(self, req, **kwargs):
        data = self.request.data

        try:
            serializer = self.get_serializer(data=data)
            serializer.is_valid(raise_exception=True)
            serializer.save()

            res = {
                "message": "Testimony Added Successfully",
                "success": True
            }
            return Response(res, status=status.HTTP_201_CREATED)
        except Exception as e:
            print(e)
            res = {
                "message": "Something went wrong",
                "success": False,
                "error": str(e),
            }
            return Response(res, status=status.HTTP_400_BAD_REQUEST)

models.py

from django.db import models


def set_image_path(instance, filename):
    return f"testimonies/{instance.name}/{filename}"


class Testimonies(models.Model):
    name = models.CharField(max_length=200)
    message = models.TextField()

    image = models.ImageField(upload_to=set_image_path)

    position = models.CharField(max_length=200)
    company = models.CharField(max_length=300)

    def __str__(self):
        return self.name

Edit: Also adding the settings of the Cloudfront

Please help me with this!! This is the first time ever I've run into such kind of bug in production... Thanks a lot for your time and insights!!

1. I tried using SerializerMethodField for the image field in serializers.py, but with that I wasn't able to upload image.
2. I tried changing the MEDIA_URL in the settings.py
3. I tried looking into the code of the library, there I saw it does add https://, but it adds it infront of the custom_domain which I set in storages.py to CLOUDFRONT_DOMAIN. I'll paste that specific code from the library too here.

    def url(self, name, parameters=None, expire=None, http_method=None):
        # Preserve the trailing slash after normalizing the path.
        name = self._normalize_name(clean_name(name))
        params = parameters.copy() if parameters else {}
        if expire is None:
            expire = self.querystring_expire

        if self.custom_domain:
            url = "{}//{}/{}{}".format(
                self.url_protocol,
                self.custom_domain,
                filepath_to_uri(name),
                "?{}".format(urlencode(params)) if params else "",
            )

            if self.querystring_auth and self.cloudfront_signer:
                expiration = datetime.utcnow() + timedelta(seconds=expire)
                return self.cloudfront_signer.generate_presigned_url(
                    url, date_less_than=expiration
                )

            return url

That's all I tried, with this it wasn't showing it on localhost but the issue still persisted on the production server.

My frontend is hosted on vercel. Although I don't think that is a problem, because the response itself has flaw, so frontend isn't doing anything to it, that I'm sure of!

Answer 1

Your static url should be like this

STATIC_URL = f'{AWS_CLOUDFRONT_DOMAIN}/{STATIC_LOCATION}/'