Long polling with Ajax with sleep()/time_sleep_until() in while() loop

Question

I'm willing to set up a long polling Ajax call to check for orders in my e-commerce web app. There is a specificity in this application in the way that customers are able to place order in the future. As such, in the admin panel, we have past orders and futures orders (that can be 2 months or 20 minutes in the future).

Basically, I want the admin user in the back-end to be warned as soon as a future order comes to an end (the future date reaches the current time). To proceed, I make the user admin doing an Ajax call (as soon as they are connected to the admin) to the server to check for futures orders to arrive. This Ajax call is a long polling request as the call waits for the server to deliver result. If server has nothing to offer, the request keeps pending until there is an order to show.

Ajax request

    (function poll() {
        setTimeout(function() {
            $.ajax({
                url: '{{ path('commande_check') }}',
                method: 'post',
                success: function(r) {
                    if(r.ids) alert('New order!'); // I've simplified this part of the code to make it clean, admin are actually warned through Node.JS server
                },
                error: function() {},
                complete: poll
            });
        }, 5000);
    })();

{{ path('commande_check') }} (edited from Edit2)

public function checkAction(Request $request)
{

    if($request->isXmlHttpRequest())
    {
        $response = new Response();
        $em = $this->getDoctrine()->getManager();

        $ids = array();
        while(!$ids)
        {
            $ids = $em->getRepository('PaymentBundle:Commande')->findNewestOrders(new \DateTime());
            if($ids)
                break;
            else
                time_sleep_until(time() + self::SECONDS_TO_SLEEP);
        }

        if($ids)
        {
            return new JsonResponse(array(
                'ids' => $ids
            ));
        }
        $response->setStatusCode(404);
        return $response;
    }

    $response = new Response();
    $response->setStatusCode(405);
    return $response;
}

findNewestOrder() method

public function findNewestOrders(\DateTime $datetime)
{
    $query = $this->createQueryBuilder('c')
                  ->select('c.id')
                  ->leftJoin('Kt\PaymentBundle\Entity\Paiement', 'p', \Doctrine\ORM\Query\Expr\Join::WITH, 'p.id = c.paiement')
                  ->andWhere('p.etat = 0')
                  ->where("DATE_FORMAT(c.date, '%Y-%m-%d %H:%i') = :date")
                  ->setParameter('date', $datetime->format('Y-m-d H:i'))
                  ->andWhere('c.kbis IS NULL')
                  ->andWhere('c.notif = 0')
                  ->getQuery();

    return $query->getArrayResult();
}

My problem is the alert sometimes never get shown whereas the record in the DB gets updated. The weirdest things is it sometimes happens even when I've leaved the page making the Ajax call like if it keeps running in the background. I think the problem comes from the time_sleep_until() function. I tried with sleep(self::SECOND_TO_SLEEP) but the problem was the same.

Any help would by gladly appreciated. Thanks!

Edit 1

I sense there is something to do with connection_status() function as the while loop appears to continue even if the user has switched page causing the field notif to be updated in the background.

Edit 2

As per my answer, I've managed to overcome this situation but the problem still remains. The admin does get the notification properly. However, I do know the Ajax call still keeps going on as the request has been made.

My problem is now: could this result in a server resources overload? I'm willing to start a bounty on this one as I'm eager to know the best solution to achieve what I want.

Answer 1

I think I got it all wrong.

The intent of long-polling Ajax is not that there is only one connection that stays opened such as websockets (as I thought it did). One would have to make several requests but much less than regular polling.

• Regular polling

The intent for Ajax regular polling is one makes a request to the server every 2 or 3 seconds to have a semblance of real-time notification. These would result in many Ajax calls during one minute.

• Long polling

As the server is waiting for new data to be passed on to the browser, one would need to make only a minimal number of requests per minute. As I'm checking in the database for new order every minute, using long polling can make me lower the number of requests per minute to 1.

• In my case

In consequence, the specificity of the application makes the use of Ajax long-polling unnecessary. As soon as a MySQL query has been made for a specific minute, there is no need for the query to run again in the same minute. That means I can do regular polling with an interval of 60000 ms. There's also no need to use sleep() nor time_sleep_until().

Here's how I ended up doing it:

JS polling function

    (function poll() {
        $.ajax({
            url: '{{ path('commande_check') }}',
            method: 'post',
            success: function(r) {
                if(r.ids)
                    alert('New orders');
            },
            error: function() {},
            complete: function() {
                setTimeout(poll, 60000);
            }
        });
    })();

{{ path('commande_check') }}

public function checkAction(Request $request)
{

    if($request->isXmlHttpRequest())
    {
        $em = $this->getDoctrine()->getManager();
        $ids = $em->getRepository('PaymentBundle:Commande')->findNewestOrders(new \DateTime());

        if($ids)
        {
            return new JsonResponse(array(
                'ids' => $ids
            ));
        }
        $response = new Response();
        $response->setStatusCode(404);
        return $response;
    }
    $response = new Response();
    $response->setStatusCode(405);
    return $response;
}

As such, I end up with one request per minute that will check for new orders.

Thanks @SteveChilds, @CayceK and @KevinB for their kind advice.

Answer 2

I have finally managed to overcome this bug but without digging deeply in the problem.

I have separated the code that updates the notif field from the code that fetch new orders. In that way, the while loop still goes on but cannot update the field.

The field is therefore updated on success of the first ajax call by making a new ajax request to update the field. Therefore, the admin always receives the notification.

I just have to enquiry on a memory/thread level to see what consumption of resources this loop uses.

As no solution has been found despites my workaround for the initial bug, I won't accept my answer as the problem remains still.

Many thanks for all the help on that question.

Answer 3

I personally would check frequently rather than have one request which runs for a long time - its not really ideal to have long running processes like this as they tie up server connections and really, its just bad practice. Plus the browser may well time the connection out, which is why you may not be seeing the responses you expect.

Have you tried changing the ajax call so it calls in say, every 60 seconds (or however often you want), checks for new orders since the last time it was polled (simply keep a track of this in the page / HTML5 local storage so it persists across pages and pass it in the ajax request as a parameter) and then simply returns an indication of yes there have been new orders, or no there hasn't?

You can then display a message if there have been new orders.

Answer 4

In general for this problem it is kinda rough to say. We don't have a lot of information as to exactly what your other functions do. Like findNewestOrders...

We can assume that it pulls all new orders that have yet to be fulfilled by the admin and therefore will be displayed. However, if it is looking only for orders that are exactly equal they will never be filled.

Theoretically this will run forever if no new order ever is filed. You have no time limit on this so it is possible that the server feels like you have a case in which while will never be false and executes an exceeded execution time.

As per your comment

time_sleep_until

Returns TRUE on success or FALSE on failure.

The only way it would ever fail is if the function itself failed or some server side issue caused a failure return. As you never officially visit the page and no act of leaving your ajax'd page submits a failure response it should never really fail.

I think it might be more wise to look into doing a CRON job for this and have a database of incomplete orders that you query instead. The CRON can run every minute and populate the database. The run on the server would not be that great as it would most likely take no more than 30 seconds any way.

Long-polling may be a great idea for many functions, but I'm not wholly confident it is in the case. I would seriously recommend setInterval as the load on the server and client would not be that great in a 30 seconds call every minute or two. That is your call in the end.