CODEWITHSUNDEEP
phpcurlweb-scrapinglibcurl
John
John

Reputation: 177

How to get website using cURL

I am not able to fetch data using cURL in php AS I can do this using curl command in terminal like below - 

curl http://www.universalapplianceparts.com/search.aspx?find=w10130694

This command provide expected result in terminal.

I want same result in php using curl

Below is my php code - 

$url = "http://www.universalapplianceparts.com/search.aspx?find=W10130694";
$ch = curl_init();  
    curl_setopt($ch, CURLOPT_URL,"http://www.universalapplianceparts.com/");
    curl_setopt($ch, CURLOPT_POST, true);  
    curl_setopt($ch, CURLOPT_HEADER, true);
    curl_setopt($ch, CURLOPT_POSTFIELDS, $url); 
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true); 
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
    $output = curl_exec ($ch); 
    curl_close ($ch); 
    var_dump($output);

Please let me know why I am not getting same result using php curl as I am getting in terminal.

Thanks

Upvotes: 4

Views: 6707

Answers (3)

Ravindra Bhalothia
Ravindra Bhalothia

Reputation: 1770

I tried your code and getting error related to length so you need to add content length as well in your curl request. Try below code hope it will help you : 

$url = "http://www.universalapplianceparts.com/search.aspx?find=W10130694";
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch, CURLOPT_HTTPHEADER, array("Content-Type: application/text"));
curl_setopt($ch, CURLOPT_USERAGENT, array("Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1a2pre) Gecko/2008073000 Shredder/3.0a2pre ThunderBrowse/3.2.1.8"));
curl_setopt($ch, CURLOPT_HTTPHEADER, array('Content-Length: 0'));
curl_setopt($ch, CURLOPT_POST, 1);
$result = curl_exec($ch);
var_dump($result);

Upvotes: 0

Exception
Exception

Reputation: 787

Try this ::

    $url = "http://www.universalapplianceparts.com/search.aspx?find=W10130694";
$ch1= curl_init();
curl_setopt ($ch1, CURLOPT_URL, $url );
curl_setopt($ch1, CURLOPT_HEADER, 0);
curl_setopt($ch1,CURLOPT_VERBOSE,1);
curl_setopt($ch1, CURLOPT_USERAGENT, 'Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; .NET CLR 1.0.3705; .NET CLR 1.1.4322; Media Center PC 4.0)');
curl_setopt ($ch1, CURLOPT_REFERER,'http://www.google.com');  //just a fake referer
curl_setopt($ch1, CURLOPT_RETURNTRANSFER, 1);
curl_setopt($ch1,CURLOPT_POST,0);
curl_setopt($ch1, CURLOPT_FOLLOWLOCATION, 20);

$htmlContent= curl_exec($ch1);
echo $htmlContent;

Upvotes: 7

irfanengineer
irfanengineer

Reputation: 1300

Try This:

 $url = "http://www.universalapplianceparts.com/search.aspx?find=W10130694";
                $options = Array(
    CURLOPT_RETURNTRANSFER => TRUE,  
    CURLOPT_FOLLOWLOCATION => TRUE,  
    CURLOPT_AUTOREFERER => TRUE,
    CURLOPT_HTTPHEADER => array("Content-Type: application/text"),
    CURLOPT_TIMEOUT => 120,  
    CURLOPT_MAXREDIRS => 10,
    CURLOPT_USERAGENT => "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1a2pre) Gecko/2008073000 Shredder/3.0a2pre ThunderBrowse/3.2.1.8",
    CURLOPT_URL => $url,  
    );

    $ch = curl_init(); 

    curl_setopt_array($ch, $options);  

    $data = curl_exec($ch);

    curl_close($ch);

Upvotes: 0

Related Questions