Reputation: 442
Is there a maximum number of fetch requests that can be made in a single call to URLFetchApp.fetchAll()
? I have limited experience with AppScripts and could not find anything on the docs indicating that there is a limit on the number of requests, but I'm curious to see if a limit does exist. Any input is welcome.
Upvotes: 3
Views: 1509
Reputation: 15385
Expanding on the research undertaken by Tanaike in his benchmarking research for UrlFetchApp.fetchAll()
. I was able to make 2199 requests successfully, but more triggered a rate limit on the endpoint so I was unable to test furhter.
After reading the aforementioned benchmark, I took to Apps Script to see if I could find a limit for the method. I used httpbin for testing, as this is what is provided by Google in the example snippet for fetchAll(requests)
:
// Make both a POST request with form data, and a GET request.
var resumeBlob = Utilities.newBlob('Hire me!', 'text/plain', 'resume.txt');
var formData = {
'name': 'Bob Smith',
'email': '[email protected]',
'resume': resumeBlob
};
// Because payload is a JavaScript object, it is interpreted as
// as form data. (No need to specify contentType; it defaults to either
// 'application/x-www-form-urlencoded' or 'multipart/form-data')
var request1 = {
'url': 'https://httpbin.org/post',
'method' : 'post',
'payload' : formData
};
// A request may also just be a URL.
var request2 = 'https://httpbin.org/get?key=value';
UrlFetchApp.fetchAll([request1, request2]);
I modified this by initialising an array of length n
and populating it with requests like so:
function fetchAllTester() {
var resumeBlob = Utilities.newBlob('Hire me!', 'text/plain', 'resume.txt');
var formData = {
'name': 'Bob Smith',
'email': '[email protected]',
'resume': resumeBlob
}
var request1 = {
'url': 'https://httpbin.org/post',
'method' : 'post',
'payload' : formData
}
let requests = new Array(1000).fill(request1)
UrlFetchApp.fetchAll(requests).map(x => console.log(x.getContentText()))
}
I started with 1000 as this has already been confirmed to work. The results were then as follows:
HTTP 502
returned by httpbin.HTTP 502
returned by httpbin.HTTP 502
returned by httpbin.At this point sending 2199 requests seemed to be intermittantly responsive, but with an HTTP 502: Bad Gateway error which pointed to the problem coming from outside of the fetchAll()
method.
It appears that is a limit on UrlFetchApp.fetchAll()
exists, then it is high.
From the Quotas for Google Services page, the only limit on calls that is listed is:
Feature Consumer (e.g., gmail.com) and G Suite free edition (legacy) Google Workspace accounts URL Fetch calls 20,000 / day 100,000 / day
My assumption therefore, based on the testing I have done and the documentation provided by Google, is that the method is limited only by regular Apps Script Quotas; that is to say, 20,000 requests per day for consumer accounts and 100,000 requests per day for Workspace accounts.
Upvotes: 6