CODEWITHSUNDEEP
javascriptpythonajaxdjango
Giorgio
Giorgio

Reputation: 2287

Django: Launch a server-side computation and dynamically update the client

I am creating a web application with Django (back end) and JavaScript (front end).

The main scenario is the following: a client asks the server to train a neural network (which can take quite a lot: seconds or even minutes, sometimes). Immediately after sending the request, the client shows to the user how the training session is going; it does so by retrieving and displaying the training error, which is actually just a number.

For training the network I made this function, which I implemented as a generator:

def train(network, trainset):
    # [setup variables...]
    while (current_error > target_error):
        # [continue to train the network...]
        # [update current_error...]
        yield current_error

On the front end (a simple HTML page) I have a JS script for retrieving data dynamically from a Django view. My Django view goes something like this:

def getError(request):
    # ...
    generator = train(net, dataset)
    return StreamingHttpResponse(generator)

At the moment I can retrieve the data with an AJAX call but I always have to wait for the server to train the whole neural network before I can see the training errors.

My problem is that I would like the user to view the error updates while the server is training the network, not after: that's why I implemented the training function as a generator (and that's why I chose to return a StreamingHttpResponse).

What can I do for retrieving the data while the server is computing? Specifically, I would like to update the front end each time the training generator yields a number.

Upvotes: 1

Views: 644

Answers (2)

John Moutafis
John Moutafis

Reputation: 23134

Initial analysis:

Let's have a look at the documentation:

  1. yield:

    The yield expression is used when defining a generator function

  2. generator:

    A function which returns a generator iterator. It looks like a normal function except that it contains yield expressions for producing a series of values usable in a for-loop or that can be retrieved one at a time with the next() function.

From the aforementioned it is clear that your code sample works well but not as intended:

The train() function yields a generator which gets "transmitted" to your front-end from the getError after the whole calculation is done.

Reform the problem based on analysis:

With the above mentioned, your problem will be to get the calculations of each step of the train() process and display them to the front-end.

A skeleton of a solution:

  • Create a global variable (yes I said it) list.

  • Refactor train() function to use the previous list:

    train_errors = []

def train(network, trainset):
    # [setup variables...]
    while (current_error > target_error):
        # [continue to train the network...]
        # [update current_error...]
        train_errors.append(current_error)

  • Assuming that train() is located in views.py file, refactor the getError() to return the train_errors list:

    from django.http import JsonResponse

def getError(request):
    # ...
    return JsonResponse(train_errors, safe=False)

    I utilize JsonResponse, but feel free to use anything that suites you.

  • Make your front-end to repeat the AJAX call every N seconds, by utilizing javascript's setInterval, and display the differences from the previous state of the data.
    Example of use:

    var myIntervalCall = setInterval(myAJAXcall, 500);

function myAJAXcall{
  // Create your AJAX call in here 
  // And sufficient code to display the changes on the received data.
}

    The above code will make myAJAXcall to the back-end, every 0.5 sec

The above will allow you to make an initial approach to your problem and you can/must expand on this (barbaric) solution.

EDIT/Disclaimer (due to @George and @brunodesthuilliers comments):

The above, is not a good solution!! It is only a quick solution to get the author "unstuck" from his current problem and help him move on with his development.
The "solution" provides the basis for a thought process, considering the problem at hand (the actual problem that the author has at the moment) and therefore provides a minimal and initial approach to a solution.

The aforementioned distiled: DO NOT TAKE THIS SOLUTION AS CANON! it is only a "jump start".

Upvotes: 2

bruno desthuilliers
bruno desthuilliers

Reputation: 77912

Solution 1: use celery for the computation, storing progress in redis, and have your front side js code poll a view that will get progress from a dedicated view.

Solution 2: use a websocket.

Upvotes: 1

Related Questions