CODEWITHSUNDEEP
google-apps-scripturlfetch
The Swift Coder
The Swift Coder

Reputation: 442

Maximum Number of Request Parameters for fetchAll()

Is there a maximum number of fetch requests that can be made in a single call to URLFetchApp.fetchAll()? I have limited experience with AppScripts and could not find anything on the docs indicating that there is a limit on the number of requests, but I'm curious to see if a limit does exist. Any input is welcome.

Upvotes: 3

Views: 1509

Answers (1)

Rafa Guillermo
Rafa Guillermo

Reputation: 15385

Answer:

Expanding on the research undertaken by Tanaike in his benchmarking research for UrlFetchApp.fetchAll(). I was able to make 2199 requests successfully, but more triggered a rate limit on the endpoint so I was unable to test furhter.

Research:

After reading the aforementioned benchmark, I took to Apps Script to see if I could find a limit for the method. I used httpbin for testing, as this is what is provided by Google in the example snippet for fetchAll(requests):

// Make both a POST request with form data, and a GET request.
var resumeBlob = Utilities.newBlob('Hire me!', 'text/plain', 'resume.txt');
var formData = {
  'name': 'Bob Smith',
  'email': '[email protected]',
  'resume': resumeBlob
};
// Because payload is a JavaScript object, it is interpreted as
// as form data. (No need to specify contentType; it defaults to either
// 'application/x-www-form-urlencoded' or 'multipart/form-data')
var request1 = {
  'url': 'https://httpbin.org/post',
  'method' : 'post',
  'payload' : formData
};
// A request may also just be a URL.
var request2 = 'https://httpbin.org/get?key=value';
UrlFetchApp.fetchAll([request1, request2]);

I modified this by initialising an array of length n and populating it with requests like so:

function fetchAllTester() {
  var resumeBlob = Utilities.newBlob('Hire me!', 'text/plain', 'resume.txt');
  var formData = {
    'name': 'Bob Smith',
    'email': '[email protected]',
    'resume': resumeBlob
  }

  var request1 = {
    'url': 'https://httpbin.org/post',
    'method' : 'post',
    'payload' : formData
  }

  let requests = new Array(1000).fill(request1)

  UrlFetchApp.fetchAll(requests).map(x => console.log(x.getContentText()))
}

I started with 1000 as this has already been confirmed to work. The results were then as follows:

  • 2000 requests - worked
  • 2500 requests - HTTP 502 returned by httpbin.
  • 2001 requests - worked
  • 2100 requests - worked
  • 2200 requests - HTTP 502 returned by httpbin.
  • 2199 requests - worked
  • 2199 requests - HTTP 502 returned by httpbin.

At this point sending 2199 requests seemed to be intermittantly responsive, but with an HTTP 502: Bad Gateway error which pointed to the problem coming from outside of the fetchAll() method.

Conclusion:

It appears that is a limit on UrlFetchApp.fetchAll() exists, then it is high.

From the Quotas for Google Services page, the only limit on calls that is listed is:

Feature Consumer (e.g., gmail.com) and G Suite free edition (legacy) Google Workspace accounts
URL Fetch calls 20,000 / day 100,000 / day

My assumption therefore, based on the testing I have done and the documentation provided by Google, is that the method is limited only by regular Apps Script Quotas; that is to say, 20,000 requests per day for consumer accounts and 100,000 requests per day for Workspace accounts.

References:

Upvotes: 6

Related Questions