CODEWITHSUNDEEP
phperror-handlingcronbatch-processing
LeeTee
LeeTee

Reputation: 6601

check cron job has run script properly - proper way to log errors in batch processing

I have set up a cronjob to run a script daily. This script pulls out a list of Ids from a database, loops through each to get more data from the database and geneates an XML file based on the data retrieved.

This seems to have run fine for the first few days, however, the list of Ids is getting bigger and today I have noticed that not all of the XML files have been generated. It seems to be random IDs that have not run. I have manually run the script to generate the XML for some of the missing IDs individually and they ran without any issues.

I am not sure how to locate the problem as the cron job is definately running, but not always generating all of the XML files. Any ideas on how I can pin point this problem and quickly find out which files have not been run.

I thought perhaps add timestart and timeend fields to the database and enter these values at the start and end of each XML generator being run, this way I could see what had run and what hadn't, but wondered if there was a better way.

set_time_limit(0);

//connect to database
$db = new msSqlConnect('dbconnect');

$select = "SELECT id FROM ProductFeeds WHERE enabled = 'True' ";

$run = mssql_query($select);
while($row = mssql_fetch_array($run)){

    $arg = $row['id'];
    //echo $arg . '<br />';
    exec("php index.php \"$arg\"", $output);
    //print_r($output);

}

Upvotes: 0

Views: 788

Answers (2)

LeeTee
LeeTee

Reputation: 6601

Turned out the apache was timing out. Therefore nothing to do with using a function or the exec() function.

Upvotes: 0

Derk Arts
Derk Arts

Reputation: 3460

My suggestion would be to add some logging to the script. A simple

error_log("Passing ID:".$arg."\n",3,"log.txt");

Can give you some info on whether the ID is being passed. If you find that that is the case, you can introduce logging to index.php to further evaluate the problem.

Btw, can you explain why you are using exec() to run a php script? Why not excute a function in the loop. This could well be the source of the problem.

Because with exec I think the process will run in the background and the loop will continue, so you could really choke you server that way, maybe that's worth trying out as well. (I think this also depends on the way of outputting:

Note: If a program is started with this function, in order for it to continue running in the background, the output of the program must be redirected to a file or another output stream. Failing to do so will cause PHP to hang until the execution of the program ends.

Maybe some other users can comment on this.

Upvotes: 1

Related Questions