CODEWITHSUNDEEP
amazon-web-servicesamazon-s3copy
Sachin Jain
Sachin Jain

Reputation: 21842

Copy content from one S3 bucket to another S3 bucket with different keys

I have two S3 buckets with two different set of access and secret keys. Here is the set:

Bucket1, Key1, Secret1

And

Bucket2, Key2, Secret2, Token

I am trying to trigger S3-S3 copy via aws cli like this:

aws s3 cp s3://key1:secret1@Bucket1 s3://key2:secret2@Bucket2

I have few questions:

What would be the best approach to achieve this use case ?

Upvotes: 6

Views: 2625

Answers (1)

John Rotenstein
John Rotenstein

Reputation: 269490

To copy a file between Amazon S3 buckets, you must use credentials that have permission to access both buckets, or apply a bucket policy to the destination bucket that permits the access.

It is not possible to specify two sets of credentials because the AWS Command-Line Interface (CLI) is only calling a single AWS API, which performs the copy 'from' the source bucket directly to the destination bucket. The AWS CLI does not download the object -- it simply tells S3 to copy the object to another bucket (which can even be in a different region).

Therefore, you should create a bucket policy on the destination bucket that permits the credentials being used (User or Role) to PutObject into the destination bucket.

The policy would be similar to:

{
    "Version": "2012-10-17",
    "Id": "Policy1",
    "Statement": [
        {
            "Sid": "Stmt1",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::ACCOUNT-NUMBER:role/ROLE-NAME"
            },
            "Action": "s3:PutObject",
            "Resource": "arn:aws:s3:::DESTINATION-BUCKET/*"
        }
    ]
}

The above is assuming the command is being called from an Amazon EC2 instance with an assigned role. To call from User credentials, use:

"AWS": "arn:aws:iam::ACCOUNT-NUMBER:user/USER-NAME"

Upvotes: 5

Related Questions