CODEWITHSUNDEEP
phpcurl
Sheel Ranjan
Sheel Ranjan

Reputation: 167

How to combine multiple cURL requests in one?

This is the code I am currently using

function curl_get_contents($url) {
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_HEADER, 0);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
    curl_setopt($ch, CURLOPT_URL, $url);
    $data = curl_exec($ch);
    return $data;
}

function meta_scrap($filename, $other, $programming) {
    $link = 'https://graph.facebook.com/?id=' . $filename . '&scrape=true&method=post';
    $output = curl_get_contents($link);
    $output = json_decode($output);
    $ogtitle = $output->title;
}

I call meta_scrap($filename); 8 times on a single webpage. This makes the page load really slow. Is there something that I can do about it? I read about curl_multi_init() I tried to use it like this

function curl_get_contents($pages) {
    $ch = curl_init();
    $ch = array();
    $mh = curl_multi_init();
    for ($i = 0; $i < count($pages); $i++) {
        $page = $pages[$i];
        $ch[$i] = curl_init();
        curl_setopt($ch, CURLOPT_HEADER, 0);
        curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
        curl_setopt($ch, CURLOPT_URL, $page);
        curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1);
        curl_multi_add_handle($mh, $ch[$i]);
    }

    $running = 0;
    do {
        curl_multi_exec($mh, $running);
    } while ($running > 0);

    $data = curl_exec($ch);
    $results = reset(json_decode(curl_multi_getcontent($ch[$i]), true));
    $resultCount = count($results);

    curl_close($ch);
    return $data;
}

With this I get no output? Could anyone help me modify my code so that it gives correct output?

Upvotes: 0

Views: 3128

Answers (1)

symcbean
symcbean

Reputation: 48387

You seem to have started working with multiple requests then cut and pasted a single curl request handler here:

$data = curl_exec($ch);
$results = reset(json_decode(curl_multi_getcontent($ch[$i]), true));
$resultCount = count($results);

....but I can't imagine where you would have found piece of code which uses reset() like this.

You try to put the response into $results, yet you then throw this array away and return something completely different.

Try this....

...
$running = count($pages);
do {
   curl_multi_exec($mh, $running);
   usleep(5000);
} while ($running > 0);

$responses=array();
for ($i = 0; $i < count($pages); $i++) {
   $responses[$i]=json_decode(curl_multi_getcontent($ch[$i]), true);
}
return $responses;

If you will be reusing the function then you should also remove and close each curl handle then close the multi-handle before returning.

See also my recent blog post about curl_multi_exec().

Upvotes: 1

Related Questions