CODEWITHSUNDEEP
phpcurlcron
Dylan Cross
Dylan Cross

Reputation: 5986

Crontab Linux with PHP, MySQL, and Curl

I am trying to run some php with curl inside of it every so often so that it will retrieve information from a website and then insert it into my database. I am able to get the cronjob to work, and the php works because I have it to send me an email every time it runs, and it does. What's not working however is the CURL script, and or the MySQL scripts.

Cronjob:

* * * * * /usr/bin/php -q  /var/www/rstracker/cronjobs.php

cronjobs.php:

<?php 
$to      = '';
$subject = 'RSTracker Cronjob Ran!';
$message = 'Successful cronjob!';
$headers = 'From: ' . "\r\n" .
    'Reply-To: ' . "\r\n" .
    'X-Mailer: PHP/' . phpversion();

mail($to, $subject, $message, $headers);
?>

<?php 
include("connect.php");

$query = mysql_query("SELECT id FROM users");
while($row = mysql_fetch_array($query))
{

    $userID = $row['id'];
    $date = mktime();

    // create a new cURL resource
    $ch = curl_init();

    // set URL options
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
    curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)");
    curl_setopt($ch, CURLOPT_URL, "http://services.runescape.com/m=hiscore_oldschool/index_lite.ws?player=".$username);
    curl_setopt($ch, CURLOPT_HEADER, 0);

    // grab HTML
    $data = curl_exec($ch);

    // close cURL resource, and free up system resources
    curl_close($ch);

    //echo "<textarea>$data</textarea><br />";


    $hs = explode("\n",$data);
    $skills = array("Overall","Attack","Defence","Strength","Constitution","Ranged","Prayer","Magic","Cooking","Woodcutting","Fletching","Fishing","Firemaking","Crafting","Smithing","Mining","Herblore","Agility","Thieving","Slayer","Farming","Runecrafting","Hunter","Construction");
    $i = 0;
    for($i = 0; $i<count($skills);$i++) {
     // Explode each skill into 3 values - rank, level, exp
     $stat = explode(',', $hs[$i]);
     $out[$skills[$i]] = Array();
     $out[$skills[$i]]['rank'] = $stat[0];
     $out[$skills[$i]]['level'] = $stat[1];
     $out[$skills[$i]]['xp'] = $stat[2];
    }




    foreach($out as $key => $value)
    {
        if($value['level']>0||$value['xp']>0||$value['rank']>0)
        {
            mysql_query("INSERT INTO skills SET userID='$userID', skill='$key', level='".$value["level"]."', xp='".$value["xp"]."', rank='".$value["rank"]."', date='$date'");
        }
    }

}


?>

Upvotes: 0

Views: 509

Answers (2)

Marc B
Marc B

Reputation: 360702

Most likely connect.php isn't in the proper directory. When running as a CRON job, a php script's "working directory" is the home directory of the account it's running under. Unless "connect.php" is somewhere in the include_path, or in the same directory, it's almost certainly NOT being found.

Since it's not being loaded, you don't get a connection to the database.

Since all of your database and curl code are simply assuming that life is perfect, they're not set up to handle the failure of having no DB connection.

As well, even if all this other stuff was working ok, nowhere in your code are you setting $username, so the URL you're scraping off the runescape site is merely

http://services.runescape.com/m=hiscore_oldschool/index_lite.ws?player=

so you'd be scraping invalid data anyways.

General tip: You do not need to create a new curl object each time. you can create ONE curl object outside the loop, then reset the URL inside the loop.

Upvotes: 1

hek2mgl
hek2mgl

Reputation: 158030

I expect that the problem is because of this line:

include("connect.php");

Note that the working directory of the php script when it get's executed by crond is the file sytem root: / rather than the directory where the php script is stored.

Change the path to:

include(__DIR__ . '/connect.php');

Upvotes: 1

Related Questions