Reputation: 61
I want to ask something make me confuse. I'am using CURL to get html code from this link
echo set_user_agent_grab("https://www.bandros.co.id/produk/dress-atasan-baju-rajut-wanita-sad-500");
And This is my function
function set_user_agent_grab($link){
$headers = ["text/html; charset=UTF-8"];
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $link);
curl_setopt($ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.2 (KHTML, like Gecko) Chrome/5.0.342.3 Safari/533.2');
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_MAXREDIRS, 10);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 30);
curl_setopt($ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_ENCODING, 'gzip');
$result = curl_exec($ch);
curl_close($ch);
return $result;
}
The problem, sometimes i got return empty, i dont know this is from my server or from the site protection with i dont know, please tell me, thank you.
Upvotes: 0
Views: 1126
Reputation: 21483
CURLOPT_VERBOSE should reveal what happened. so just check if curl_exec fail, and if it does, throw a RuntimeException, then, next time it happens, check your php error logs. additionally you can check what curl_errno() and curl_error says.
function set_user_agent_grab($link) {
$headers = [
"text/html; charset=UTF-8"
];
$ch = curl_init ();
$debugfileh = tmpfile ();
$debugfile = stream_get_meta_data ( $debugfileh ) ['uri'];
try {
curl_setopt ( $ch, CURLOPT_VERBOSE, 1);
curl_setopt ( $ch, CURLOPT_STDERR, $debugfileh);
curl_setopt ( $ch, CURLOPT_URL, $link );
curl_setopt ( $ch, CURLOPT_USERAGENT, 'Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.2 (KHTML, like Gecko) Chrome/5.0.342.3 Safari/533.2' );
curl_setopt ( $ch, CURLOPT_FOLLOWLOCATION, true );
curl_setopt ( $ch, CURLOPT_RETURNTRANSFER, true );
curl_setopt ( $ch, CURLOPT_MAXREDIRS, 10 );
curl_setopt ( $ch, CURLOPT_CONNECTTIMEOUT, 30 );
curl_setopt ( $ch, CURLOPT_HTTP_VERSION, CURL_HTTP_VERSION_1_1 );
curl_setopt ( $ch, CURLOPT_HTTPHEADER, $headers );
curl_setopt ( $ch, CURLOPT_ENCODING, 'gzip' );
$result = curl_exec ( $ch );
if (! is_string ( $result )) {
$errstr = "curl_exec failed: " . curl_errno ( $ch ) . ": " . curl_error ( $ch ) . ". debuginfo: " . file_get_contents ( $debugfile );
throw new RuntimeException ( $errstr );
}
return $result;
} finally{
fclose ( $debugfileh );
curl_close ( $ch );
}
}
Upvotes: 1