CODEWITHSUNDEEP
phpcurl
user2524169
user2524169

Reputation: 267

cURL Multi Threading?

I've heard a lot about php's multi threading with cURL but have never really tried it and I find it a bit tough to understand how it actually works. Could anyone convert this into curl_multi?

$path1 = array("path1", "path2", "path3"); //example
$path2 = array("path1", "path2", "path3"); //example
$opt = curl_init($path1);
curl_setopt($opt, CURLOPT_RETURNTRANSFER, true);
$content = curl_exec($opt);
curl_close($opt);
file_put_contents($path2, $content);

What I want to actually do is to download multiple files from the arrays path 1 into path 2 using curl_multi.

Upvotes: 0

Views: 16454

Answers (2)

Mani
Mani

Reputation: 441

The above accepted answer is outdated/wrong, So, correct answer has to be up voted.

http://php.net/manual/en/function.curl-multi-init.php

Now, PHP supports fetching multiple URLs at the same time.

There is a very good function written by someone, http://archevery.blogspot.in/2013/07/php-curl-multi-threading.html

This is the function: 

function runRequests($url_array, $thread_width = 4) {
$threads = 0;
$master = curl_multi_init();
$curl_opts = array(CURLOPT_RETURNTRANSFER => true,
    CURLOPT_FOLLOWLOCATION => true,
    CURLOPT_MAXREDIRS => 5,
    CURLOPT_CONNECTTIMEOUT => 15,
    CURLOPT_TIMEOUT => 15,
    CURLOPT_RETURNTRANSFER => TRUE);
$results = array();

$count = 0;
foreach($url_array as $url) {
    $ch = curl_init();
    $curl_opts[CURLOPT_URL] = $url;

    curl_setopt_array($ch, $curl_opts);
    curl_multi_add_handle($master, $ch); //push URL for single rec send into curl stack
    $results[$count] = array("url" => $url, "handle" => $ch);
    $threads++;
    $count++;
    if($threads >= $thread_width) { //start running when stack is full to width
        while($threads >= $thread_width) {
            usleep(100);
            while(($execrun = curl_multi_exec($master, $running)) === -1){}
            curl_multi_select($master);
            // a request was just completed - find out which one and remove it from stack
            while($done = curl_multi_info_read($master)) {
                foreach($results as &$res) {
                    if($res['handle'] == $done['handle']) {
                        $res['result'] = curl_multi_getcontent($done['handle']);
                    }
                }
                curl_multi_remove_handle($master, $done['handle']);
                curl_close($done['handle']);
                $threads--;
            }
        }
    }
}
do { //finish sending remaining queue items when all have been added to curl
    usleep(100);
    while(($execrun = curl_multi_exec($master, $running)) === -1){}
    curl_multi_select($master);
    while($done = curl_multi_info_read($master)) {
        foreach($results as &$res) {
            if($res['handle'] == $done['handle']) {
                $res['result'] = curl_multi_getcontent($done['handle']);
            }
        }
        curl_multi_remove_handle($master, $done['handle']);
        curl_close($done['handle']);
        $threads--;
    }
} while($running > 0);
curl_multi_close($master);
return $results;
}

You can just use it.

Upvotes: 3

TeaCupApp
TeaCupApp

Reputation: 11452

This is nice project to start with...

https://github.com/jmathai/php-multi-curl

I am using curl multi and it is awesome indeed. I am using this to make faster push notifications.

https://github.com/Krutarth/FlashSnsPns

Upvotes: 2

Related Questions