CODEWITHSUNDEEP
phpapicurl
Lorenzo99
Lorenzo99

Reputation: 23

api call using php cURL not returning all pages

I'm trying to hit an api and populate a database table with the returned data. There are over 9000 rows of data I need to get, and only 100 rows of data per page. I'm setting up loops, to control which page I'm at, but only the first 100 records (first page) gets returned.

My code currently is only grabbing two pages for testing.

I'm using PHP cURL to accomplish this

for($pageLoop = 0; $pageLoop < 2; $pageLoop++){
  $curl2 = curl_init();
  curl_setopt_array($curl2, array(
    CURLOPT_URL => "https://SOME_SERVER/rest/search/?api_key=MY_API_KEY&searchTerm=&pageNumber=".$pageLoop."&pageSize=100",
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_TIMEOUT => 30,
    CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
    CURLOPT_CUSTOMREQUEST => "GET",
    CURLOPT_HTTPHEADER => array(
      "cache-control: no-cache"
    ),
  ));

  $response2 = curl_exec($curl2);
  $err2= curl_error($curl2);

  if(!curl_exec($curl2)){
    die('Error: "' . curl_error($curl2) . '" - Code: ' . curl_errno($curl2));
  }
  curl_close($curl2);

  for($itemLoop = 0; $itemLoop <  100; $itemLoop++){
    echo 'chpl product number yy: ' . $response['results'][$itemLoop]['chplProductNumber'] . '<br>';
  }//END ITEM LOOP        
}//END PAGE LOOP

Expected results are a list of items for each page. There are 97 pages total as of today and I'm only getting the first page. Thanks

Upvotes: 0

Views: 467

Answers (1)

Jonathan
Jonathan

Reputation: 510

My guess is, you are calling page 0 which doesn't return any results. So then on your second iteration, you are calling page 1 for the first time.

Hence, only getting first page result.

Try this:

$pageLoop = 1;
do {
  $cl = curl_init();
  curl_setopt_array($cl, array(
    CURLOPT_URL => "https://SOME_SERVER/rest/search/?api_key=MY_API_KEY&searchTerm=&pageNumber=".$pageLoop."&pageSize=100",
    CURLOPT_RETURNTRANSFER => true,
    CURLOPT_TIMEOUT => 30,
    CURLOPT_HTTP_VERSION => CURL_HTTP_VERSION_1_1,
    CURLOPT_CUSTOMREQUEST => "GET",
    CURLOPT_HTTPHEADER => array(
      "cache-control: no-cache"
    ),
  ));

  $response = curl_exec($cl);
  curl_close($cl);

  for($itemLoop = 0; $itemLoop <  100; $itemLoop++)
    echo 'chpl product number yy: ' . $response['results'][$itemLoop]['chplProductNumber'] . '<br>';

  $pageLoop++;
} while ($pageLoop <= 2);

Also, I notice a few variable names which are completely unused. for example $err2 and $response2. My latest edit cleans that up a bit.

Upvotes: 1

Related Questions