Reputation: 34556
I'm trying to scrape the below feed (with permission) via PHP cURL:
http://www.safc.com/Home/RSS Feeds/News%20Feed
Loads fine in a browser, but gives me a 400 'bad request' with cURL.
$ch = curl_init($uri); //http://www.safc.com/Home/RSS Feeds/News%20Feed
curl_setopt_array($ch, array(
CURLOPT_RETURNTRANSFER => 1,
CURLOPT_ENCODING => '',
CURLOPT_TIMEOUT => CURL_CONNECT_TIMEOUT,
CURLOPT_USERAGENT => CURL_USER_AGENT,
CURLOPT_SSL_VERIFYPEER => false,
CURLOPT_FOLLOWLOCATION => true
));
$ret = curl_exec($ch);
Result is a 400; I know this from looking in curl_getinfo()
.
CURL_USER_AGENT
is an arbitrary identifier as I realised some other feeds wouldn't spit out content unless this header was present. I have tried removing the headers one by one, and tried adding a few more, but that approach feels a bit needle/haystack.
Before I approach the owners of the site, does anyone know how I might resolve this?
Upvotes: 2
Views: 669
Reputation: 917
use http://www.safc.com/home/rss%20feeds/news%20feed check different between "Home" and "home" there is 301 redirect when you use "Home".
Upvotes: 1