CODEWITHSUNDEEP
phpcurlcurl-multi
pmm
pmm

Reputation: 654

curl_multi_select always blocking for time out value

When running Example #1 from PHP on Windows XP PHP 5.3.5 the curl_multi_select() line will always block for the duration of the timeout specified (if blank it will block for 1 sec, if I specify 5 sec timeout it will block for 5 sec) regardless of the time it takes to get the content. I suspect it's related to this bug.

The question is: what's the best work around? The best I can come up with is to get rid of curl_multi_select() and usleep(x) as a way to save some cycles.

Upvotes: 3

Views: 3784

Answers (1)

Butifarra
Butifarra

Reputation: 1114

This might help as long as you can live with 1 second blocking.

There is a comment on the manual page for curl_multi_select that mentions this blocking lasts until at least one connection has completed or $timeout seconds, whichever occurs first. They also write that calls to curl_multi_select should be wrapped:

private function full_curl_multi_exec($mh, &$still_running) {
        do {
                $rv = curl_multi_exec($mh, $still_running);
        } while ($rv == CURLM_CALL_MULTI_PERFORM);
        return $rv;
}

Then modify your loop that checks for running handles:

// execute the handles
$still_running = null;
$this->full_curl_multi_exec($mh, $still_running);

// check whether the handles have finished
do { // "wait for completion"-loop 
        curl_multi_select($mh); // non-busy (!) wait for state change 
        $this->full_curl_multi_exec($mh, $still_running); // get new state
        while ($info = curl_multi_info_read($mh)) { 
            // process completed request (e.g. curl_multi_getcontent($info['handle'])) 
        }
} while ($still_running);

Prior to this modification, code that was tested with PHP 5.4 was not working on an Amazon instance running PHP 5.3.20 due to this bug in PHP 5.3.18, which causes calls to curl_multi_select() to never return anything other than -1

I am now able to retrieve data from nearly 1,300 URLs in less than 30 seconds using batches of 200 handles.

Upvotes: 3

Related Questions