server sent events not updating until script is finished

Question

I have a small script that updates a database. It takes about a minute or so to process currently, but the time will go up as the database goes up. I want to provide updates to the user as the script runs so that they know that the status of the script and that it is still going. I implemented Server-Side-Events and while it works, none of the updates are posted to the website until the script is completely finished.

I can upload the entire script file, but here are the relevant parts.

First, at the start of the script I check to see if the script is run from the command line and if not I send the headers as shown:

$cli = (PHP_SAPI === 'cli') ? 1 : 0;
if(!$cli){
    header( 'Content-Type: text/event-stream' );
    header( 'Cache-Control: no-cache' );
}

Here is my function to send an sse_message when called

function sse_message( $type, $message, $progress=0 )
{
    $d = array("type" => $type, "msg" => $message , "progress" => $progress);
    $id = time();

    echo "id: $id" . PHP_EOL;
    echo "data: " . json_encode($d) . PHP_EOL;
    echo PHP_EOL;

    ob_flush();
    flush();
}

And finally here is my javascript implementation of the listener:

if( typeof(EventSource) !== "undefined" ){
    source = new EventSource( "../priv3/script/prowip_process.php" );
    source.addEventListener("message", function(e)
    {
        var result = JSON.parse(e.data);

        switch(result.type){
            case "update":
                if( result.msg != "" )
                    document.getElementById("prowip_status").value += "\n[INFO]: "+result.msg;
                document.getElementById("prowip_progress").value = result.progress;
                break;

            case "success":
                document.getElementById("prowip_status").value += "\n[COMPLETED]:"+result.msg;
                document.getElementById("prowip_progress").value = 100;
                document.getElementById("button_prowip_exit").disabled = false;
                source.close();
                break;

            case "error":
                document.getElementById("prowip_status").value += "\n[ERROR]: "+result.msg;
                document.getElementById("button_prowip_exit").disabled = false;
                source.close();
                break;      
        }
    }, false);
}
else{
    document.getElementById("prowip_status").value += "\n\n[ERROR]: Your browser does not support server-sent events.  Please upgrade your browser or use a different browser for this function.  Please note this script will not work with Internet Explorer."
    document.getElementById("button_prowip_exit").disabled = false;
}

Again - everything works perfect except that no updates occur during the script process and then when it finishes all of the messages and progress bar is updated.

I have tried this in several browsers, computers, etc. but none of the updates to the webpage occur until the entire script is done.

Any help or comments would be greatly appreciated. Thanks.

EDITED 4/3/2015 TO POST MORE CODE AS REQUESTED

These 3 functions call sse_message or just echo for CLI.

function show_error( $cli, $message )
{
    if ($cli)
        echo $message."\n";
    else
        sse_message( "error", $message );
}

function show_message( $cli, $message, $progress )
{
    if ($cli)
        echo $message."[".$progress."%]\n";
    else
        sse_message( "update", $message, $progress );
}

function show_success( $cli, $message )
{
    if ($cli)
        echo $message."[100%]\n";
    else
        sse_message( "success", $message );
}

Here is a clip out of the main function that shows how these functions are called:

foreach ( $projects as $number => $project ) {
            $cn = strtok($number, "." );
            $pn = strtok(".");
            $wipover = $project['wip'] - $project['wip30'] - $project['wip60'] - $project['wip90'];
            $arover = $project['ar'] - $project['ar30'] - $project['ar60'] - $project['ar90'];
            $process_counter++;
            if( $process_counter >= $check ){
                $progress+=5;
                $check += intval ( $counter / 16 );
                show_message($cli,"",$progress);
            }
            if( ($project['empty']==1) and ($project['wip']==0) and ($project['ar']==0) ){
                //Do nothing...
            }
            else{
                $write_counter++;
                $update_query = "update projects set wip='{$project['wip']}', wip30='{$project['wip30']}', wip60='{$project['wip60']}', wip90='{$project['wip90']}', wipover='$wipover', ar='{$project['ar']}', ar30='{$project['ar30']}', ar60='{$project['ar60']}', ar90='{$project['ar90']}', arover='$arover' where client_num='$cn' and new_num='$pn'";
                if( ! $db->query( $update_query ) ){
                    show_error( $cli, "ERROR - update query: [$update_query] returned error: [".$db->error."]", $progress );
                }
            }
        }

Answer 1

I had the same issue. It turned out to be the web server caching the output from php.

Within PHP ob_flush() and flush() forces the output from PHP to the web server, but cannot force the web server to push it out to the client.

My solution was to pad the output to 800,000 characters before flushing from PHP. i.e:

echo "data: " . str_pad($message, 800000) . PHP_EOL . PHP_EOL;

Each response back to the client is approx 6Kb big, so it's not very efficient but it was the only way I could get it to work.

Hope that helps!