CODEWITHSUNDEEP
phpcurlphp-curl
abc
abc

Reputation: 35

Is there any way to speed up cURL

I was wondering whether there's any way to speed up a cURL request like this via PHP.

<?php
$ch = curl_init();

curl_setopt($ch, CURLOPT_URL, 'https://www.randomsite.com/path');
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_IPRESOLVE, CURL_IPRESOLVE_V4);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.79 Safari/537.36');
$html = trim(curl_exec($ch));

echo $html;

curl_close($ch);
?>

I'm looping this a few hundred times and it's taking pretty long, therefore I'd like to know if there's any way to speed up this process.

Upvotes: 0

Views: 3650

Answers (1)

hanshenrik
hanshenrik

Reputation: 21463

yeah, first thing you can do, is to enable compression, if the content you're fetching benefits from compression (this excludes jpg/png/gif/anything pre-compressed, but stuff like html/css/javascript/xml benefits greatly from transfer compression) - set CURLOPT_ENCODING to emptystring to have curl take care of transfer compression automatically.

I'm looping this a few hundred times and it's taking pretty long - unless you cannot for some reason fetch them concurrently, just use the curl_multi api, then you can fetch hundreds of them simultaneously, should be significantly faster than using the curl_exec method (which can only fetch them serially)

Upvotes: 2

Related Questions