PHP script overloads the server

Question

I have to run a php script daily, the code is this:

<?php
// SET PROCESS PRIORITY

// SETTING UP THE LOG FILE
$ob_file = fopen('sync.log','w');

function ob_file_callback($buffer)
{
  global $ob_file;
  fwrite($ob_file,$buffer);
}


ob_start('ob_file_callback');

// GET PARAMETERS
$lastid=$_GET['lastid'];
    if ($lastid && !is_numeric($lastid)){
        die("Loser");
    }

// SERVER ROOT CONFIGURATION
$serverpath = 'http://dev.xxx.com/ops/';

// FACEBOOK SDK
require '../classes/facebook.php';
require '../classes/fbconfig.php';


// MYSQL CONFIG
include('../dbconfig.php');

// OPEN DATABASE CONNECTION $CON


$con = mysql_connect($dbhost,$dbuser,$dbpass);
mysql_select_db($dbname, $con);


// GET LAST ID FROM DATABASE AND COUNT TO ZERO
// ONLY 10 PER TIMES
$sql = "SELECT * FROM ops_pictures
ORDER BY id DESC
LIMIT 10;";

$query = mysql_query($sql,$con);

while($row = mysql_fetch_array($query)) {

    if (!$lastid){  
        $lastid = $row['id'];
    }
}

//START COUNTING
for ($i = $lastid; $i >= 0 ; $i --)
    {

// set_time_limit(); // RESET TIMEOUT TO GO FOREVER

echo $i .' - ';

// LOAD RECORDS FROM FACEBOOK AND DISPLAY THE PHOTO URL

// Get User ID
$user = $facebook->getUser();

// Login or logout url will be needed depending on current user state.
if ($user) {
  $logoutUrl = $facebook->getLogoutUrl();
} else {
  $loginUrl = $facebook->getLoginUrl(
   array(
    'scope'      => 'email,offline_access,user_likes,read_stream,publish_stream,user_about_me,user_photos'
    )
 );
};

// FQL QUERY FOR THE PICTURE
$actualurl = $serverpath . 'photo.php?pid=' . $i;
//echo $actualurl;

try {
        $fql = 'SELECT url, normalized_url, share_count, like_count, comment_count, total_count,
commentsbox_count, comments_fbid, click_count FROM link_stat WHERE url="'.$actualurl.'"';
        $ret_obj = $facebook->api(array(
                                   'method' => 'fql.query',
                                   'query' => $fql,
                                 ));

        // FQL queries return the results in an array, so we have
        //  to get the user's name from the first element in the array.
        $linkstat = $ret_obj[0];

        $theurl = $linkstat['url'];
        $sharecount = $linkstat['share_count'];
        $likecount = $linkstat['like_count'];
        $commentcount = $linkstat['comment_count'];
        $totalcount = $linkstat['total_count'];

      } catch(FacebookApiException $e) {
        // If the user is logged out, you can have a 
        // user ID even though the access token is invalid.
        // In this case, we'll get an exception, so we'll
        // just ask the user to login again here.
        $login_url = $facebook->getLoginUrl(); 
        echo 'Please <a href="' . $login_url . '">login.</a>';
        error_log($e->getType());
        error_log($e->getMessage());
      }   




// PUT THE RECORDS IN
$sql = "UPDATE ops_pictures SET likecount='$likecount', sharecount='$sharecount', totalcount ='$totalcount'
 WHERE id='$i'";

mysql_query($sql,$con);


//END COUNT
}

// CLOSE DATABASE
mysql_close($con);
ob_end_flush();

?>

Basically it is a cycle that for every picture in my website takes and FQL query from Facebook and store the result in my MYSQL database. I even makes a log of the files done.

Problems: - if i call it from the browser it locks up the server and any php file i try to load waits forever and gives a 500 server error. - i don't want it to timeout because i want it to cycle all the pictures in the database and refresh them with the new values

Resolutions - Cron Job (this should allow the script to run in "background" mode right? - Split the script in different parts processing 10 pictures per time (not good) - Use proc_nice() before the code to assign a lower priority to the whole php instructions.

Actually i have this script overloading the server and anything gets unusable, what do you think about that ?

Thanks a lot !!

Answer 1

Run task manager or top on your server, see if it is hitting your CPU or your Memory or both.

If it is hitting your memory, use unset and clean up your variables as you go and don't need them any longer, through each loop. http://php.net/manual/en/function.unset.php

If you are running into CPU usage issues, you can use nice if you're on a unix or the task manager I believe in windows has something similar. Can I throttle the max CPU usage of a php script?

The other bottleneck can be hard drive throughput but I doubt that is your problem.

The program will take a long time to run, but it will finish eventually.

You can also try to optimize your code and reuse as much as possible, use references instead of just assignment as that makes a copy doubling memory usage.

$myvar = &$other_var['var'];

Be sure to set your memory usage high, something like:

ini_set('memory_limit', '700M');

And to set your timeout to forever like you have above:

set_time_limit (0);