CODEWITHSUNDEEP
phpcurl
Rakesh Shetty
Rakesh Shetty

Reputation: 4578

CURL is not properly fetching the data from a site

Im fetching the data from a site www.example.com.The data is in table like structure and there is also a pagination for that.im getting first page data properly and for fetching next pages data im running my code in forloop.i know total no pages which will be 3.My code is as follows :-

$url = "http://www.example.com/browseall";

$ch = curl_init();

curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch,CURLOPT_URL,$url);

$output = curl_exec($ch);

$html = new simple_html_dom();
$html->load_file($url);
foreach($html->find('div.full_listing_pager') as $pages)
{
    $page = $pages->children(2)->plaintext;
}
curl_close($ch);
$limit = $page+1;

echo "limit--->".$limit;


echo "<table border=1>";
echo "<tr>";
echo "<th>Listing Id </th>";
echo "<th>Free Km Allowed</th>";
echo "<th>Free Days allowed</th>";
echo "<th>Driver requirements</th>";
echo "<th>Owner comments</th>";
echo "</tr>";


for($i=1;$i<$limit+1;$i++) //$limit =3(no of pages)
{
    $url=urlencode('http://www.example.com.au/browseall?browse_filter[from_city]=0&browse_filter[to_city]=0&browse_filter[car]=0&browse_filter[by_date]=0&page='.$i);
    $ch = curl_init();
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
    curl_setopt($ch, CURLOPT_URL, $url);

    $output = curl_exec($ch);

    foreach($html->find('table.full_listings_table tbody tr.more_info_second') as $div)
    {
        $str = "<tr>";

        $data = $div->find('td p b',0)->plaintext;
        $str .="<td>".$data."</td>";

        $data = $div->find('td div b',0)->plaintext;
        $str .="<td>".$data."</td>";

        $data = $div->find('td br b',0)->plaintext;
        $str .="<td>".$data."</td>";

        $data = $div->find('td div',0)->plaintext;
        $dataLen = strlen($data);
        $temp = "Driver requirements:";
        $tempLen = strlen($temp);
        $pos = strpos($data,$temp,0);

        $sum = $pos + $tempLen;
        $finalData = substr($data,$sum,$dataLen-$sum);

        $str .="<td>".$finalData."</td>"; 

        $data = $div->find('td div',2)->plaintext;
        $data = str_replace("Owner comments:"," ",$data);
        $str .="<td>".$data."</td>"; 

        echo $str."</tr>";
    }
}
echo "</table>";
curl_close($ch);

Problem is im getting the first page data 3 times .I want all 3 pages data in a table.Is there is something wrong in my code ?please help me on this, im new to cURL.

Upvotes: 1

Views: 624

Answers (1)

Rakesh Shetty
Rakesh Shetty

Reputation: 4578

God finally got the solution ...really im such an idiot i wrote

$url=urlencode('http://www.example.com.au/browseall?browse_filter[from_city]=0&browse_filter[to_city]=0&browse_filter[car]=0&browse_filter[by_date]=0&page='.$i);

there is no need of using urlencode.Now by writing this following code i got my solution:-

$url='http://www.example.com.au/browseall?browse_filter[from_city]=0&browse_filter[to_city]=0&browse_filter[car]=0&browse_filter[by_date]=0&page='.$i;

Upvotes: 1

Related Questions