CODEWITHSUNDEEP
pythondjangohttpchunked-encodinghttp-chunked
Zubair Afzal
Zubair Afzal

Reputation: 2086

Django return http early flush (chunked response)

I have an algorithm which waits for almost 5 seconds to generate response and I want to send an ack (http 200) to user as soon as he sends request to tell him that his request has been received and wait for 5 seconds.

Generator function:

def chunked_res():
    yield "Chunk 1"

    stop = time.time() + 5    # wait for 5 seconds
    while time.time() < stop:
        pass

    yield "Chunk 2"

And in some view:

response = HttpResponse ( chunked_res() )
response['Connection'] = 'close'
response['Transfer-Encoding'] = 'chunked'
response['status'] = 200
return response

Response in browser:

"Transfer-Encoding: chunked\nstatus: 200\nConnection: close\nContent-Type: text/html; charset=utf-8\n\nChunk 1Chunk 2"

Problem: I am getting required response(Chunk 1, Chunk 2) but after 5 seconds. I want to send "Chunk 1" first and then "Chunk 2" after 5 seconds(update respose). Are there any particular settings/changes to implement this?

Update:

Django = 1.4 python = 2.7

Upvotes: 7

Views: 5452

Answers (4)

krubo
krubo

Reputation: 6406

Just clarifying the other answers: for Django 1.5+, you must use StreamingHttpResponse. Regular HttpResponse will collect the full response and return it only when finished.

from django.http import StreamingHttpResponse

def chunked_res():
    yield "Chunk 1"
    yield " " * 1024  # Encourage server to begin transferring and browser to begin rendering

    time.sleep(5)     # wait for 5 seconds

    yield "Chunk 2"

def myview(request):
    return StreamingHttpResponse(chunked_res())

Upvotes: 1

Zubair Afzal
Zubair Afzal

Reputation: 2086

Actually the solution was to make first chunk size of at least 1024 character for browser to show incrementally.

How to stream an HttpResponse with Django

def chunked_res():
    yield "Chunk 1"
    yield " " * 1024  # Encourage browser to render incrementally (either 1024 or 1024-7{length of "chunk 1"} = 1017)

    time.sleep(5)  # wait for 5 seconds

    yield "Chunk 2"


def myview(request):
    g = chunked_res()
    return HttpResponse(g)

If you are using nginx then you have to set proxy_buffering=off, for server to flush response as 1024 data chunk ready. http://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_buffer_size

Works with: HTTP/1.0 and HTTP/1.1

Upvotes: 3

monkut
monkut

Reputation: 43830

They just added StreamingHttpResponse in django 1.5, and by passing an iterator in earlier versions (<= 1.4) , you can stream ( or "chunk"), data.

def chunked_res():
    yield "Chunk 1"

    time.sleep(5)  # wait for 5 seconds

    yield "Chunk 2"


def myview(request):
    g = chunked_res()
    return HttpResponse(g)

Upvotes: 2

David Houlder
David Houlder

Reputation: 59

You really want to avoid doing this...

stop = time.time() + 5    # wait for 5 seconds
while time.time() < stop:
    pass

It will cause the CPU usage of your process to spike to 100% for 5 seconds, which is a huge amount for a web application. In a shared hosting environment you risk getting a nasty email from your hosting provider, and if you've got your own server, those cycles you're eating up in the loop could be usefully deployed in handling other requests. Do this instead:

time.sleep(5)

That will put the thread (or process) to sleep for 5 seconds, and the kernel is then free to schedule some other task, or sleep the CPU (saving power and cooling costs) if there's nothing else to do.

Upvotes: 2

Related Questions