CODEWITHSUNDEEP
amazon-web-servicesamazon-s3
Bostonian
Bostonian

Reputation: 687

How can I avoid AWS S3 throttling issues?

All my work to access AWS S3 is in the region: us-east2 and the AZ is us-east-2a.

But I saw some throttling complaints from S3, so I am wondering if I move some of my work to another AZ like us-east-2b, could it mitigate the problem? ( Or it will not help since us-east-2a and us-east-2b are actually pointing to same endpoint?)

Thank you.

Upvotes: 4

Views: 15120

Answers (1)

Chris Williams
Chris Williams

Reputation: 35258

The throttling is not per AZ, its for a bucket. The below quote is from the AWS documentation.

You can send 3,500 PUT/COPY/POST/DELETE and 5,500 GET/HEAD requests per second per partitioned prefix in an S3 bucket. When you have an increased request rate to your bucket, Amazon S3 might return 503 Slow Down errors while it scales to support the request rate. This scaling process is called partitioning.

To avoid or minimize 503 Slow Down responses, verify that the number of unique prefixes in your bucket supports your required transactions per second (TPS). This helps your bucket leverage the scaling and partitioning capabilities of Amazon S3. Additionally, be sure that the objects and the requests for those objects are distributed evenly across the unique prefixes. For more information, see Best Practices Design Patterns: Optimizing Amazon S3 Performance.

If possible enable exponential backoff to retry in the S3 bucket. If the application uploading to S3 is performance sensitive the suggestion would be to handoff to a background application that can upload to S3 at a later time.

Upvotes: 9

Related Questions