Does the Google Analytics API throttle requests?

Question

Does the Google Analytics API throttle requests?

We have a batch script that I have just moved from v2 to v3 of the API and the requests go through quite well for the first bit (50 queries or so) and then they start taking 4s or so each. Is this Google throttling us?

Answer 1

While Matthew is correct, I have another possibility for you. Google analytics API cashes your requests to some extent. Let me try and explain.

I have a customer / site that I request data from. While testing I noticed some strange things.

1. the first million rows results would come back with in an acceptable amount of time.
2. after a million rows things started to slow down we where seeing results returning in 5 times as much time instead of 5 minutes we where waiting 20 minutes or more for the results to return.

Example:

Request URL : https://www.googleapis.com/analytics/v3/data/ga?ids=ga:34896748&dimensions=ga:date,ga:sourceMedium,ga:country,ga:networkDomain,ga:pagePath,ga:exitPagePath,ga:landingPagePath&metrics=ga:entrances,ga:pageviews,ga:exits,ga:bounces,ga:timeOnPage,ga:uniquePageviews&filters=ga:userType%3D%3DReturning+Visitor;ga:deviceCategory%3D%3Ddesktop&start-date=2014-05-12&end-date=2014-05-22&start-index=236001&max-results=2000&oauth_token={OauthToken} Request Time (seconds:milliseconds): :0:484

Request URL : https://www.googleapis.com/analytics/v3/data/ga?ids=ga:34896748&dimensions=ga:date,ga:sourceMedium,ga:country,ga:networkDomain,ga:pagePath,ga:exitPagePath,ga:landingPagePath&metrics=ga:entrances,ga:pageviews,ga:exits,ga:bounces,ga:timeOnPage,ga:uniquePageviews&filters=ga:userType%3D%3DReturning+Visitor;ga:deviceCategory%3D%3Ddesktop&start-date=2014-05-12&end-date=2014-05-22&start-index=238001&max-results=2000&oauth_token={OauthToken} Request Time (seconds:milliseconds): :7:968

I did a lot of testing stopping and starting my application. I couldn't figure out why the data was so fast in the beginning then slow later.

Now I have some contacts on the Google Analytics Development team the guys in charge of the API. So I made a nice test app, logged some results showing my issue and sent it off to them. With the question Are you throttling me?

They where also perplexed, and told me there is no throttle on the API. There is a flood protection limit that Matthew speaks of. My Developer contact forwarded it to the guys in charge of the traffic.

Fast forward a few weeks. It seams that when we make a request for a bunch of data Google cashes the data for us. Its saved on the server incase we request it again. By restarting my application I was accessing the cashed data and it would return fast. When I let the application run longer I would suddenly reach non cashed data and it would take longer for them to return the request.

I asked how long is data cashed for, answer there was no set time. So I don't think you are being throttled. I think your initial speedy requests are cashed data and your slower requests are non cashed data.

Email back from google:

Hi Linda,

I talked to the engineers and they had a look. The response was basically that they thinks it's because of caching. The response is below. If you could do some additional queries to confirm the behavior it might be helpful. However, what they need to determine is if it's because you are querying and hitting cached results (because you've already asked for that data). Anyway, take a look at the comments below and let me know if you have additional questions or results that you can share.

Summary from talking to engineer: "Items not already in our cache will exhibit a slower retrieval processing time than items already present in the cache. The first query loads the response into our cache and typical query times without using the cache is about 7 seconds and with using the cache is a few milliseconds. We can also confirm that you are not hitting any rate limits on our end, as far as we can tell.

To confirm if this is indeed what's happening in your case, you might want to rerun verified slow queries a second time to see if the next query speeds up considerably (this could be what you're seeing when you say you paste the request URL into a browser and results return instantly)."

-- IMBA Google Analytics API Developer --

Answer 2

Google's Analytics API does have a rate limit per their docs: https://developers.google.com/analytics/devguides/reporting/core/v3/coreErrors

However they should not caused delayed requests, rather the request should be returned with a response of: 403 userRateLimitExceeded

Description of that error: Indicates that the user rate limit has been exceeded. The maximum rate limit is 10 qps per IP address. The default value set in Google Developers Console is 1 qps per IP address. You can increase this limit in the Google Developers Console to a maximum of 10 qps.

Google's recommended course of action: Retry using exponential back-off. You need to slow down the rate at which you are sending the requests.