Reputation: 127
I'm using AWS CLI to download big file from S3 bucket ( heroku db backup ) to my EC2 instance and upload big file ( about 110gb) to my AWS S3 bucket from my AWS EC2
Problem is when download ( from bucket that I don't own ) or upload to my bucket ( they are all in same region us-east-1 ) . Speed when download/upload start at about 60MB/s but decrease to 7-8MB/s after first 15gb even when I enable transfer acceleration .
So this is problem with AWS CLI config or my EC2 instance ( I'm testing with t2.micro ) ?Tks
Upvotes: 2
Views: 5248
Reputation: 15754
Here's a list of items you can try:
- Use enhanced networking on the EC2 instance.
- Use parallel workloads for the data transfer.
- Customize the upload configurations on the AWS Command Line Interface (AWS CLI).
- Use an Amazon Virtual Private Cloud (Amazon VPC) endpoint for Amazon S3.
- Upgrade your EC2 instance type.
- Use chunked transfers.
Read more here: https://aws.amazon.com/premiumsupport/knowledge-center/s3-transfer-data-bucket-instance/
Upvotes: 3