CODEWITHSUNDEEP
pythondjango
ElpieKay
ElpieKay

Reputation: 30868

Print subprocess stdout line by line in real time in Django StreamHttpResponse

I wrote an API in Django. It runs a shell script in the server and prints some logs. When the client calls the API in the browser, I want the browser to print the logs line by line. So far, the snippet is like:

from django.http import StreamingHttpResponse
import subprocess
import shlex


def test(request):
    cmd = 'bash test.sh'
    args = shlex.split(cmd)
    proc = subprocess.Popen(
        args,
        cwd='/foo/bar',
        shell=False,
        stdout=subprocess.PIPE,
        stderr=subprocess.STDOUT,
    )
    return StreamingHttpResponse(
        (line for line in iter(proc.stdout.readline, '')),
        content_type="text/plain",
    )

And test.sh is like

#!/bin/bash

echo hello
sleep 1
echo world
sleep 1
echo love
sleep 1
echo and
sleep 1
echo peace
sleep 1

The logs are not printed until bash test.sh is finished. How can I make it print the logs as if test.sh is run in the client?

I'm using Python 2.7 and Django 1.11.29. I know they are old, but I still have to use them now. Thanks for help.

Upvotes: 0

Views: 582

Answers (1)

Abdul Aziz Barkat
Abdul Aziz Barkat

Reputation: 21807

You provide the content to StreamingHttpResponse as:

(line for line in iter(proc.stdout.readline, ''))

Here you make an iterable and at iterate over it at the same time and return a generator. Although this won't really matter and you should be getting real time output. The problem may simply be that the output is too fast to actually see the real time difference or perhaps your configuration is incorrect (See Streaming HTTP response, flushing to the browser), Also another problem can be that you are not using the response as a stream with javascript and instead are downloading it fully before using it. You can try using time.sleep() to see if it is just too fast or instead a configuration problem:

import time


def iter_subproc_with_sleep(proc):
    for line in iter(proc.stdout.readline, ''):
        time.sleep(1)
        yield line

def test(request):
    cmd = 'bash test.sh'
    args = shlex.split(cmd)
    proc = subprocess.Popen(
        args,
        cwd='/foo/bar',
        shell=False,
        stdout=subprocess.PIPE,
        stderr=subprocess.STDOUT,
    )
    return StreamingHttpResponse(
        iter_subproc_with_sleep(proc),
        content_type="text/plain",
    )

Upvotes: 2

Related Questions