CODEWITHSUNDEEP
phpcurlweb-scrapingfile-get-contents
SanJeet Singh
SanJeet Singh

Reputation: 1321

Why is using cURL giving me a 403 forbidden error when file_get_contents works?

This is my PHP code:

$ch = curl_init('http://www.iphonehacks.com/2016/08/jailbreak-iphone-ipad-ios-9-3-3-pangu-1-1.html'); 
curl_setopt($ch, CURLOPT_USERAGENT, "User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36");
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); 
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); 
curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, false);

$data = curl_exec($ch);
curl_close($ch);

echo $data;

The code above gives me a 403 error but when I use file_get_contents, I can echo the whole webpage.

$data = file_get_contents('http://www.iphonehacks.com/2016/08/jailbreak-iphone-ipad-ios-9-3-3-pangu-1-1.html');

echo $data;

Could anyone please point out what's wrong with the cURL method. I know I can use file_get_contents but I am interested in learning what went wrong in first case to get a better understanding of cURL.

Upvotes: 1

Views: 3882

Answers (1)

pah
pah

Reputation: 4778

Your CURLOPT_USERAGENT option is incorrect. Change it to:

curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36");

You shall not use the prefix "User-Agent: " on CURLOPT_USERAGENT option. The website was rejecting it because it was receiving the following request header:

User-Agent: User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36

Instead of:

User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36

Upvotes: 3

Related Questions