CODEWITHSUNDEEP
phpcurl
OZZIE
OZZIE

Reputation: 7348

cURL returns empty output from valid url - no errors reported

If you just enter the urls into the browser you can see that both work, cdon works even without javascript, have they blocked cURL somehow?

I'm trying to build a scraper to benifit legal movies online which would benifit them a whole lot, seems stupid blocking scrapers in general imho. Although I'm far from sure that's whats going on here! Might be just an error somewhere..

// Works
get_file1('http://sfanytime.com/sv-SE/Sokresultat/?field=all&q=The+Matrix', '/', 'sfanytime.html');

// Saves a blank 0 KB file
get_file1('http://downloads.cdon.com/index.phtml?action=search&search_terms=The+Matrix', '/', 'cdon.html');

function get_file1($file, $local_path, $newfilename) {
    $out = fopen($newfilename, 'wb');
    if ($out === FALSE) {
        return false;
    }      
    $ch = curl_init();             
    curl_setopt($ch, CURLOPT_FILE, $out);
    curl_setopt($ch, CURLOPT_HEADER, 0);
    curl_setopt($ch, CURLOPT_URL, $file);                  
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);

    curl_exec($ch);
    $error = curl_error($ch);
    if (strlen($error) > 0) {
            echo "<br>Error is : ". $error;
        return false;
    }
    curl_close($ch);
    return true;
}

Upvotes: 3

Views: 6872

Answers (1)

DaveRandom
DaveRandom

Reputation: 88647

You should change the line

curl_setopt($ch, CURLOPT_FAILONERROR, true);

...to...

curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);

CURLOPT_FAILONERROR will cause a "silent fail" - which from what you say, is not what you want. I have replaced this with CURLOPT_FOLLOWLOCATION, because when I visit the second URL, I get redirected to a "choose your country" type page, which will be a response with an empty body - which is why you get an empty file.

There is no problem with your code as such, simply a problem with the way you handle the response from the second URL. You don't see an error because, technically, there wasn't one.

Upvotes: 3

Related Questions