CODEWITHSUNDEEP
phpcurlstreamfread
TomSawyer
TomSawyer

Reputation: 3820

CURL slower than fread while streaming file? how to speed up CURL

I'm trying to make a script to download a part of file. Just make a test with CURL and fread, i realize that CURL during streaming process is slower than fread. Why? how to speed up curl for stream a file? i don't like to use fread , fopen because i need limited time during streaming process.

Here is my sample code.

$start = microtime(true);
$f = fopen('http://news.softpedia.com/images/news2/Debian-Turns-15-2.jpeg','r');
$response = fread($f, 3); echo $response.'<br>';
$response = fread($f, 3); echo $response.'<br>';
$response = fread($f, 3); echo $response.'<br>';
$response = fread($f, 3); echo $response.'<br>';
$response = fread($f, 3); echo $response.'<br>';

$stop = round(microtime(true) - $start, 5);
echo "{$stop}s";
exit();

fread / fopen It take only 1.1s

$start = microtime(true);
$curl = curl_init('http://news.softpedia.com/images/news2/Debian-Turns-15-2.jpeg');
curl_setopt($curl, CURLOPT_BINARYTRANSFER, 1);
curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1);

curl_setopt($curl, CURLOPT_RANGE, "0-2");
$response = curl_exec($curl);echo $response.'<br>';

curl_setopt($curl, CURLOPT_RANGE, "3-5");
$response = curl_exec($curl);echo $response.'<br>';

curl_setopt($curl, CURLOPT_RANGE, "6-8");
$response = curl_exec($curl);echo $response.'<br>';

curl_setopt($curl, CURLOPT_RANGE, "9-11");
$response = curl_exec($curl);echo $response.'<br>';

curl_setopt($curl, CURLOPT_RANGE, "12-14");
$response = curl_exec($curl);echo $response.'<br>';
curl_close($curl);

$stop = round(microtime(true) - $start, 5);
echo "{$stop}s";
exit();

curl took around 2.5s. if i take more step to download more a part of the file. curl will take more slower.

Why curl is slower? and what solution is it?

Upvotes: 4

Views: 2503

Answers (5)

poda
poda

Reputation: 1

You can try the following solution:

curl_setopt($curl, CURLOPT_RANGE, "0-2, 3-5, 6-8, 9-11, 12-14");
$response = curl_exec($curl);echo $response.'<br>';
curl_close($curl);
$stop = round(microtime(true) - $start, 5);
echo "{$stop}s";
exit();

Upvotes: 0

CreatoR
CreatoR

Reputation: 1652

Try to set Keep-Alive header before first curl_exec, for example 300 seconds in such way:

$headers = array("Keep-Alive: 300");
curl_setopt($curl, CURLOPT_HTTPHEADER, $headers);

Upvotes: 0

Muayyad Alsadi
Muayyad Alsadi

Reputation: 1593

your question is about having many partial requests or about incremental buffered reading for the same request.

The fopen/fread implementation fires a single HTTP request and read it piece by piece many times.

on the other hand the curl implementation fires many HTTP requests, one request per piece (see partial range requests). so we are comparing apples and oranges

to be fair the fopen/fread would look like this

$start = microtime(true);
$f = fopen('http://news.softpedia.com/images/news2/Debian-Turns-15-2.jpeg','r');
$response = fread($f, 3); echo $response.'<br>';
fclose($f)
$f = fopen('http://news.softpedia.com/images/news2/Debian-Turns-15-2.jpeg','r');
fseek($f, 3);
$response = fread($f, 3); echo $response.'<br>';
fclose($f)
$f = fopen('http://news.softpedia.com/images/news2/Debian-Turns-15-2.jpeg','r');
fseek($f, 6);
$response = fread($f, 3); echo $response.'<br>';
fclose($f)
$f = fopen('http://news.softpedia.com/images/news2/Debian-Turns-15-2.jpeg','r');
fseek($f, 9);
$response = fread($f, 3); echo $response.'<br>';
fclose($f)
$f = fopen('http://news.softpedia.com/images/news2/Debian-Turns-15-2.jpeg','r');
fseek($f, 12);
$response = fread($f, 3); echo $response.'<br>';
fclose($f)
$stop = round(microtime(true) - $start, 5);
echo "{$stop}s";
exit();

UPDATE: I've rewrite my answer

Upvotes: 2

Dennis C
Dennis C

Reputation: 24747

It is always slower because you added additional round-trip of HTTP calls. Every curl_exec is a single HTTP request.

Upvotes: 2

Latheesan
Latheesan

Reputation: 24116

You can use curl_getinfo to see what is taking long.

It is possible the slowness could be due to the DNS lookup by the curl library. Have you tried using IP address instead of domain for the request url?

EDIT : alternatively, you can set CURLOPT_DNS_USE_GLOBAL_CACHE to true and set CURLOPT_DNS_CACHE_TIMEOUT to a longer value (e.g. 1 hour) - it's 2minutes by default.

Src: http://php.net/manual/en/function.curl-setopt.php

Upvotes: 1

Related Questions