CODEWITHSUNDEEP
google-cloud-dataflow
Mark Shalda
Mark Shalda

Reputation: 87

Is it recommended to reduce my default requests quota for Cloud Dataflow?

The requests quota for Cloud Dataflow is set to 500,000 requests per 100 second by default, is it recommended that I lower this value?

https://console.cloud.google.com/apis/api/dataflow.googleapis.com/quotas

Upvotes: 0

Views: 43

Answers (1)

Mark Shalda
Mark Shalda

Reputation: 87

The recommended minimum for requests is 500,000 per 100 seconds for the Cloud Dataflow API. Setting this value any lower has the potential to cause problems during Dataflow job execution.

Lowering this quota beyond 500,000/100s may cause requests to fail with errors in worker logs:

"429 Too Many Requests"

The Dataflow team intends to apply a minimum floor to this quota in the future to ensure users do not lower quotas beyond what is recommended.

Upvotes: 1

Related Questions