CODEWITHSUNDEEP
djangodockernginxdjango-rest-frameworkgunicorn
KdPisda
KdPisda

Reputation: 230

How to change the host in next key in a paginated URL in django rest framework?

I have a ModelSerializer in Django Rest Framework with paginated responses. So I have deployed it with gunicorn in a docker container.

gunicorn -c gunicorn_config.py app.wsgi --bind 0.0.0.0:5000

Now the problem is in the paginated responses. The next key is something like.

next: "http://0.0.0.0:5000/admin/users/?page=2&per_page=10"

In my client-side where I am consuming these APIs, I just check the next key and fetch the next response. But since the next key has the host as 0.0.0.0:5000 hence it will cause API call failure. And the purpose is not served for the next key.

So at the moment, my API server is running in a separate docker container. Which is set up via the reverse proxy in nginx.

Upvotes: 5

Views: 3022

Answers (4)

Alexandr S.
Alexandr S.

Reputation: 1776

I updated my Nginx configuration based on the initial answer, and it works perfectly now. The path in my DRF setup has been updated to match the exact domain name value.

location / {
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header X-Forwarded-Proto $scheme;

    proxy_pass http://127.0.0.1:19077;
}

Upvotes: 0

Grant McLean
Grant McLean

Reputation: 6998

In our project, the frontend server is on a public domain (domain-a) and proxies /api requests to a backend server on a private domain (domain-b) via another proxy. Our problem appeared similar to yours in that all the URLs generated by DRF used the private hostname (domain-b). However we couldn't just use the X-Forwarded-Host header directly since the value received in this header was not correct.

Instead, we added a custom setting:

USE_X_FORWARDED_HOST = True
HOSTNAME_OVERRIDE = "domain-a"

And then added some custom middleware that put that value into the X-Forwarded-Host header:

from django.conf import settings

class HostnameOverrideMiddleware:

    def __init__(self, get_response):
        self.get_response = get_response

    def __call__(self, request):
        request.META["HTTP_X_FORWARDED_HOST"] = settings.HOSTNAME_OVERRIDE
        return self.get_response(request)

Note: the value in HOSTNAME_OVERRIDE will also need to be included in ALLOWED_HOSTS.

Upvotes: 2

KdPisda
KdPisda

Reputation: 230

So I made a custom pagination class extending PageNumberPagination

from rest_framework.pagination import PageNumberPagination
def replace_query_param(url, key, val):
    """
    Given a URL and a key/val pair, set or replace an item in the query
    parameters of the URL, and return the new URL.
    """
    (scheme, netloc, path, query, fragment) = parse.urlsplit(force_str(url))
    scheme = "https"
    netloc = "api.example.com"
    query_dict = parse.parse_qs(query, keep_blank_values=True)
    query_dict[force_str(key)] = [force_str(val)]
    query = parse.urlencode(sorted(list(query_dict.items())), doseq=True)
    return parse.urlunsplit((scheme, netloc, path, query, fragment))


def remove_query_param(url, key):
    """
    Given a URL and a key/val pair, remove an item in the query
    parameters of the URL, and return the new URL.
    """
    (scheme, netloc, path, query, fragment) = parse.urlsplit(force_str(url))
    scheme = "https"
    netloc = "api.example.com"
    query_dict = parse.parse_qs(query, keep_blank_values=True)
    query_dict.pop(key, None)
    query = parse.urlencode(sorted(list(query_dict.items())), doseq=True)
    return parse.urlunsplit((scheme, netloc, path, query, fragment))

class LargeResultsSetPagination(PageNumberPagination):
    page_size = 1000
    page_size_query_param = 'per_page'
    max_page_size = 1000

    def get_next_link(self):
        if not self.page.has_next():
            return None
        url = self.request.build_absolute_uri()
        page_number = self.page.next_page_number()
        return replace_query_param(url, self.page_query_param, page_number)

    def get_previous_link(self):
        if not self.page.has_previous():
            return None
        url = self.request.build_absolute_uri()
        page_number = self.page.previous_page_number()
        if page_number == 1:
            return remove_query_param(url, self.page_query_param)
        return replace_query_param(url, self.page_query_param, page_number)

Now I am using this pagination class in all my ViewSets

class TestViewSet(viewsets.ModelViewSet):
    permission_classes = [permissions.IsAuthenticated]

    queryset = Test.objects.all().order_by("pk")
    serializer_class = test_serializers.TestSerializer
    pagination_class = LargeResultsSetPagination
    search_fields = ['name', 'description', 'follow_up', 'follow_up_type']
    filter_backends = (filters.SearchFilter,)

And it does the job, the original inspiration https://stackoverflow.com/a/62422235/5884045

Upvotes: 2

Ken4scholars
Ken4scholars

Reputation: 6296

The next link in the DRF paginator is generated using the hostname from the request. This is how the hostname is determined in the request:

def _get_raw_host(self):
    """
    Return the HTTP host using the environment or request headers. Skip
    allowed hosts protection, so may return an insecure host.
    """
    # We try three options, in order of decreasing preference.
    if settings.USE_X_FORWARDED_HOST and (
            'HTTP_X_FORWARDED_HOST' in self.META):
        host = self.META['HTTP_X_FORWARDED_HOST']
    elif 'HTTP_HOST' in self.META:
        host = self.META['HTTP_HOST']
    else:
        # Reconstruct the host using the algorithm from PEP 333.
        host = self.META['SERVER_NAME']
        server_port = self.get_port()
        if server_port != ('443' if self.is_secure() else '80'):
            host = '%s:%s' % (host, server_port)
    return host

So, check if the HTTP_X_FORWARDED_HOST header sets the correct hostname you need and if so set USE_X_FORWARDED_HOST to True in your settings. Also make sure that the hostname you need is added to ALLOWED_HOSTS.

You could also override the get_next_link() method in the PageNumberPagination class to supply the needed host/domain name

Upvotes: 6

Related Questions