CODEWITHSUNDEEP
phpapicurlshopify
Ruchi
Ruchi

Reputation: 350

Fetch all the products (more than 50,000) using the Shopify API

I am trying to fetch all the products of a Shopify store in my app using the Shopify API. I have the following code to fetch all the products page-wise. This script is breaking at some point in time, but I am not sure where. How can I make a thorough script which will fetch all the products in one go without breaking.

Here is what I have thus far:

$ch = curl_init();
for ($i=1; $i<=1000; $i++)
{
    curl_setopt($ch, CURLOPT_URL, "https://API:PASS@store/products.json?page=$i");
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE);
    curl_setopt($ch, CURLOPT_TIMEOUT, 20);
    curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 20);
    $res = curl_exec($ch);
    $res=json_decode($res, true);
    foreach($res as $products) { // A different example
        foreach($products as $product) {
            $title=$product['title'];
        }
     }
     sleep(1); 
}
curl_close($ch);

Upvotes: 3

Views: 3325

Answers (1)

alexandresaiz
alexandresaiz

Reputation: 2766

You're probably reaching the Shopify API limit call. So you'd probably need to work with queues. Here is how the Shopify API limit calculations work:

Shopify uses a leaky bucket algorithm with a bucket size of 40 calls, and a leak rate of 2 per second.

How does that work:

  • Each call you make will increase the level of a virtual bucket level by one
  • Every half a second the bucket leaks, decreasing the current level by one
  • Your calls are processed immediately as long as the bucket has room, otherwise you get a 429 error
  • The X-Shopify-Shop-Api-Call-Limit header will be indicating the current level and the bucket size as 'x/40'

What does that mean for your app:

  • You can do bursts of up to 40 calls in a second. They will all be processed immediately
  • Once you hit the limit it will take 500 ms before another call will be allowed through
  • As long as you stay under an average of 2 calls per second you won't hit the limit
  • This actually maxes out to around 600 calls per store per 5 minutes

Upvotes: 5

Related Questions