CODEWITHSUNDEEP
shopify
Ranjit Singh
Ranjit Singh

Reputation: 1

Shopify API : Call limit

Im getting "Too many requests" 429(code) error while making burst api calls.

Here is what I'm doing.

Make 39 requests simultaneously and wait for requests to complete. If any request's response is received, immediately make new another request. With completion of any request the number of requests in shopify bucket should decrease and new made request should be processed. But I still get "Too many requests" error.

Here is the code snippet

    // aquire a permit to enter into block
    boundedSemaphore.acquire();

    // Hit the shopify api
    Response response = RestUtils.invoke(URL, operation,
            MediaType.APPLICATION_JSON_TYPE, headerMap, queryParameterMap,
            MediaType.APPLICATION_JSON_TYPE, signatureMethod);

    // Check response       
    if (!RestUtils.isValidStatusCode(response.getStatus())) {
        throw new SystemException("Status :" +response.getStatus());
    }
    if (response.getHeaderString(X_SHOPIFY_SHOP_API_CALL_LIMIT)
            .equalsIgnoreCase("39/40"))
        Thread.sleep(10000);

    // release permit for other threads to enter            
    boundedSemaphore.release();

At any given time there are no more than 39 requests active from my side.

The boundedSemaphore is java.util.concurrent.Semaphore with max 39 permits. Only 39 threads can enter this particular block at a time. With completion of any request the semaphore will release a permit and new thread will aquire it and will make a new request.

Shopify bucket size is 40 requests with the leak rate of 2 requests per second so no request should trip with "Too many requests" error.

Does anyone know what is the cause of this issue.

Upvotes: 0

Views: 8032

Answers (1)

Kris Hardy
Kris Hardy

Reputation: 548

I think that you may be misunderstanding Shopify's request throttling algorithm. They limit the maximum request rate to 2 requests per second, with a burst of up to 40 requests. Each request you send increments the number of calls in the "bucket", and they expire 1 call from the bucket every 1/2 second. This is regardless of whether the Shopify API has responded or not. So, for example, you can do an immediate burst of 40 requests, but you must then wait at least 1/2 second before sending the next request. They are purely looking at the time that has passed since the last request, and not at whether or not the connection is still open.

I also see that you have a Thread.sleep() call, but only when the X_SHOPIFY_SHOP_API_CALL_LIMIT header equals 39/40. There are 2 potential issues:

  • First, this sleep will ONLY fire when the header returns "39/40". Only that single thread will sleep, but another couple threads might immediately make other calls and receive "40/40" or get a 429 return code and you will never sleep in those cases.

  • Second, make sure that X_SHOPIFY_SHOP_API_CALL_LIMIT equals "X-Shopify-Shop-Api-Call-Limit", and that the header you are receiving from Shopify actually matches this.

https://docs.shopify.com/api/introduction/api-call-limit

Upvotes: 5

Related Questions