CODEWITHSUNDEEP
pythonamazon-web-servicesamazon-s3boto3
searcot jabali
searcot jabali

Reputation: 438

boto3 S3 limit upload speed of a large file

Is there a way to limit the available bandwidth for the Python Boto3 S3 file upload process? I am uploading some pretty heavy files (each file is approximately 5 GB in size) The upload process consumes my entire bandwidth for a while. This is causing some issues.

Is it possible to set a hard limit on how much upload speed boto3 can use? Either an absolute limit like 10 mbps or a relative limit of 10% of available capacity?

Upvotes: 0

Views: 1517

Answers (1)

jarmod
jarmod

Reputation: 78573

Investigate the configuration options available in TransferConfig, including:

The maximum number of concurrent S3 API transfer operations can be tuned to adjust for the connection speed. Set the max_concurrency attribute to increase or decrease bandwidth usage.

Upvotes: 1

Related Questions