Reputation: 21
Hello StackOverflow community, I've encountered a problem when I try to use cURL methods on PHP. I tried with this sample code:
$html_brand = "www.google.com/";
$ch = curl_init();
$options = array(
CURLOPT_URL => $html_brand,
CURLOPT_RETURNTRANSFER => true,
CURLOPT_HEADER => true,
CURLOPT_HTTPHEADER => array("User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36"),
CURLOPT_FOLLOWLOCATION => true,
CURLOPT_SSL_VERIFYPEER => false,
CURLOPT_SSL_VERIFYHOST => false,
CURLOPT_ENCODING => "",
CURLOPT_AUTOREFERER => true,
CURLOPT_CONNECTTIMEOUT => 120,
CURLOPT_TIMEOUT => 120,
CURLOPT_MAXREDIRS => 10,
);
curl_setopt_array( $ch, $options );
$response = curl_exec($ch);
$httpCode = curl_getinfo($ch, CURLINFO_HTTP_CODE);
if ( $httpCode != 200 ){
echo "Return code is {$httpCode} \n"
.curl_error($ch);
} else {
echo "<pre>".htmlspecialchars($response)."</pre>";
}
curl_close($ch);
And it always ends with an error, displaying it on screen:
Return code is 0 Recv failure: Connection was reset
Or this error, when trying to reach any site with https:
Return code is 0 Failed to connect to www.google.com port 443: Timed out
This are my settings:
Is it a code error or any server configurations I have not considered?
The HTTP_HOST
value in Apache is localhost:8080
, which I'm not really sure if it has anything to do with my problem, but maybe it's worth noting.
Thank all of you in advance.
Upvotes: 1
Views: 2412
Reputation: 4445
I've developed a custom function, that works fine for GET, POST and ajax
requests.
So here it's
function spider($header = array(), $referer = false, $url, $cookie = false,$post = false)
{
if (!$cookie)
{
$cookie = "cookie.txt";
}
$ch = curl_init();
curl_setopt($ch, CURLOPT_ENCODING, 'gzip,deflate,sdch');
if (isset($header) && !empty($header))
{
curl_setopt($ch, CURLOPT_HTTPHEADER, $header);
}
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_TIMEOUT, 200);
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_MAXREDIRS, 5);
curl_setopt($ch, CURLOPT_USERAGENT,
"Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7");
curl_setopt($ch, CURLOPT_COOKIEJAR, realpath($cookie));
curl_setopt($ch, CURLOPT_COOKIEFILE, realpath($cookie));
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);
if (isset($referer) && $referer != false)
{
curl_setopt($ch, CURLOPT_REFERER, $referer);
} else
{
curl_setopt($ch, CURLOPT_REFERER, $url);
}
//if have to post data on the server
if (isset($post) && !empty($post) && $post)
{
curl_setopt($ch, CURLOPT_POST, true);
curl_setopt($ch, CURLOPT_POSTFIELDS, $post);
} //endif
$data = curl_exec($ch);
$info = curl_getinfo($ch);
print_r($info);
curl_close($ch);
if($info['http_code'] == 200){
return ($data);
}else{
return false;
}
}
so, parameters are,
$header
=> Headers to send to server, this should be associative array
.
$referer
=> Referrer to current page(if any).
$url
=> URL that you want to get.
$cookie
=> cookie file, should be on the same level where the calling script is.
$post
=> data to post (if any)
I tested this function as follows.
echo spider(FALSE, FALSE, 'https://www.google.com/ncr');
here is header that I got.
Array
(
[url] => https://www.google.com/
[content_type] => text/html; charset=UTF-8
[http_code] => 200
[header_size] => 1665
[request_size] => 701
[filetime] => -1
[ssl_verify_result] => 20
[redirect_count] => 1
[total_time] => 2.215
[namelookup_time] => 0
[connect_time] => 0
[pretransfer_time] => 0
[size_upload] => 0
[size_download] => 9024
[speed_download] => 4074
[speed_upload] => 0
[download_content_length] => 0
[upload_content_length] => 0
[starttransfer_time] => 0.437
[redirect_time] => 1.7
[certinfo] => Array
(
)
[redirect_url] =>
)
and here is snapshot of this request.
Hope this will help you...
Upvotes: 0