CODEWITHSUNDEEP
phpdebuggingcurl
kwh
kwh

Reputation: 364

PHP cURL -randomly- not working, error debugging provides nothing

Basically, I have a simple cURL fetch webpage function that includes the following;

function FETCH_WEBPAGE($url) {

    $curl_handle = curl_init();
    curl_setopt($curl_handle, CURLOPT_URL, $url);
    curl_setopt($curl_handle, CURLOPT_CONNECTTIMEOUT, 2);
    curl_setopt($curl_handle, CURLOPT_FOLLOWLOCATION, true);
    curl_setopt($curl_handle, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($curl_handle, CURLOPT_USERAGENT, 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/534.30 (KHTML, like Gecko) Chrome/12.0.742.112 Safari/534.30');
    $file = curl_exec($curl_handle);

    if($file===false) {
        DEBUG_TXT('Error reading '.$url.chr(10).curl_error($curl_handle));
    } else {
        DEBUG_TXT('Fetching: '.$url.' - Got: '.$response['url'].chr(10).$file);
    }

    $response = curl_getinfo($curl_handle);
    curl_close($curl_handle);

    if(trim($response['url'])=='') {
        $response['url'] = $url;
    }

    return array('file'=>$file,'url'=>$response['url']);

}

Now, I'm running my own Apache server and also have a dedicated server running Apache. When I run this code from my own machine, everything works smoothly. My DEBUG_TXT returns the "Fetching / Got" (with contents) response as long as the server I'm requesting from is available. (It creates a DEBUG text file.)

However, when I run this from my dedicated server I CANNOT get it to work. I always get the error "could not find host" or "name lookup timed out" as my cURL error.

The STRANGEST THING is that for other users of the website, this function works PERFECTLY FINE and does not give them errors - even when attempting to access identical URLs. I cannot, for the life of me, even fathom why this selectively doesn't work for me and me alone.

Edit:

I should mention that curl_getinfo() seems to work for me even when curl_exec() does not. I can fetch header information, for example, perfectly fine, but curl_exec() will still fail. It's bothering me greatly as this seems to be directly related to my user account on the website - could the way that I'm fetching cookie information for myself be related to this error? I couldn't imagine how it could be, but it's quite literally the only difference (as far as the PHP interpreter is concerned) between how I'm dealing with the way I would personally use the function versus another user on the site.

Furthermore, I can successfully ping the websites I'm trying to access from my dedicated machine when connecting via SSH.

Upvotes: 0

Views: 1195

Answers (1)

kwh
kwh

Reputation: 364

I solved my problem - though I still have no idea what the original issue was. I switched to using get_headers() with stream_context_set_default() and fopen() with stream_context_create().

Unfortunately (for me), fopen() respects HTTP response codes, so it will not fetch page data like custom 404 error pages or 503 pages. My workaround for that is to check the headers before I attempt fopen, and just output the file contents as "404 - Page not found" etc.

This isn't actually a solution to my original problem, but it fixed the issue. I am still baffled as to why cURL stopped working out of nowhere. Although, I have had issues in the past with cURL selectively deciding to "not work" even on my own machine. And users of my website have reported (though infrequently) errors of the same type.

Everything's running smoothly now, though, so it seems the problem is solved.

Upvotes: 1

Related Questions