CODEWITHSUNDEEP
google-cloud-storageairflowgoogle-cloud-composer
Frans
Frans

Reputation: 1448

Hitting TooManyRequests 429 when using GoogleCloudStorageComposeOperator

When using GoogleCloudStorageComposeOperator in Google's Cloud Composer we've started hitting TooManyRequests, HTTP 429.

The rate of change requests to the object path/file.csv exceeds the rate limit. Please reduce the rate of create, update, and delete requests.

What limit are we hitting? I think it's this limit but I'm not sure:

There is a write limit to the same object name of once per second, so rapid writes to the same object name won't scale.

Does anyone have a sane way around this issue? On retry it usually works, but would be neat to not rely on it working on retry.

Upvotes: 1

Views: 4488

Answers (1)

vitooh
vitooh

Reputation: 4272

It's hard to say, without details, but this is rather Storage than Composer Issue. It is described in Troubleshooting guide for Cloud Storage.

There you can find some more references to dig more about it. On Quotas and Limit page I have found:

When a project's bandwidth exceeds quota in a location, requests to affected buckets can be rejected with a retryable 429 error or can be throttled. See Bandwidth usage for information about monitoring your bandwidth.

It seems that this error is intended to be retried, so I think implementation of try/catch mechanism might be a solution.

Upvotes: 2

Related Questions