CODEWITHSUNDEEP
amazon-web-servicesamazon-s3aws-cli
Nick Charney Kaye
Nick Charney Kaye

Reputation: 4431

AWS CLI s3 copy fails with 403 error, trying to administrate a user-uploaded object

Trying to copy a file from an S3 bucket to my local machine:

aws s3 cp s3://my-bucket-name/audio-0b7ea3d0-13ab-4c7c-ac66-1bec2e572c14.wav ./

fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden

Things I have confirmed:

Bucket policy is intended to grant wide open access:

    {
        "Sid": "AdminAccess",
        "Effect": "Allow",
        "Principal": "*",
        "Action": [
            "s3:*"
        ],
        "Resource": [
            "arn:aws:s3:::my-bucket-name",
            "arn:aws:s3:::my-bucket-name/*"
        ]
    }

How did I upload this object?

I uploaded this object using AWS Signature v4 signed upload policy from a web app in the client browser directly to AWS.

Upvotes: 12

Views: 36190

Answers (5)

shlomiLan
shlomiLan

Reputation: 716

In my case I have 3 accounts (A1, A2, A3) with 3 canonical users (canonical_user_account_A1, canonical_user_account_A2, canonical_user_account_A3) and 1 IAM role (R1) that is in A3.

Files are in a bucket in A2 and the files owner is canonical_user_account_A1 (this is on purpose). When I tried to list the files I didn't got any error, BUT when I tried to download one of them I got 

fatal error: An error occurred (403) when calling the HeadObject operation: Forbidden

I have added List and Get permissions for a R1 in the bucket policy and in the role permissions, in this case this is not enough, if the account were the bucket is not the owner it can't allow users from other account to get (download) files. So I needed to make sure that when I upload files I'm using:

    access_control_policy = {
    'Grants': [
        {
            'Grantee': {
                'ID': canonical_user_account_A2,
                'Type': 'CanonicalUser'
            },
            'Permission': 'READ'
        },
        {
            'Grantee': {
                'ID': canonical_user_account_A3,
                'Type': 'CanonicalUser'
            },
            'Permission': 'READ'
        },
    ],
    'Owner': {
        'ID': canonical_user_account_A1
    }
}

upload_extra_args = {'ACL': 'bucket-owner-full-control'}

s3_client.upload_file(file_path, bucket_name, s3_file_path, ExtraArgs=upload_extra_args)

s3_client.put_object_acl(AccessControlPolicy=access_control_policy, Bucket=bucket_name, Key=s3_file_path)

This allow both canonical_user_account_A2 and canonical_user_account_A3 to read and download the file.

Upvotes: 3

pawel.uramowski
pawel.uramowski

Reputation: 19

In my case above error appeared when machine that was trying to contact S3 had system time far from the current one. Setting a correct time helped.

Upvotes: 1

watsonic
watsonic

Reputation: 3373

I ran into a similar permissions issue when trying to download from s3 something I had uploaded previously. Turns out it has nothing to do with the bucket policy and everything to do with how your credentials are set when you upload and how you grant access privileges at time of upload. See this for more information on several ways to solve the problem.

Upvotes: 1

Nick Charney Kaye
Nick Charney Kaye

Reputation: 4431

It turns out, looking at the object properties, I can see the Owner of the OBJECT is "Anonymous," and also "Anonymous" user has full permission to this object.

I believe this is why I'm not able to access this object (I'm authenticated). Example: Since the "Anonymous" user has full permission, I am able to access via GET using a Web browser. This is functioning as designed. The S3 bucket is for uploading files which then become available for public consumption.

So when the file is POST'ed with the upload policy, the resulting owner is "Anonymous".

In this case, acl=bucket-owner-full-control should be used while uploading the object so the bucket owner can control the object. Doing this, the owner will still be "Anonymous", however, it'll give the bucket owner (me) the full permission and I should be able to access the object after that, via AWS CLI.

Note that acl=ec2-bundle-read is a default that's actually hard-coded into the latest AWS SDK. See https://github.com/aws/aws-sdk-java/blob/7844c64cf248aed889811bf2e871ad6b276a89ca/aws-java-sdk-ec2/src/main/java/com/amazonaws/services/ec2/util/S3UploadPolicy.java#L77

It was necessary to copy S3UploadPolicy.java into my own codebase (it's an entirely portable little utility class, it turns out) and modify it in order to use acl=bucket-owner-full-control. And I have verified that this affords the administration of uploaded objects via AWS CLI.

Upvotes: 6

anoop-khandelwal
anoop-khandelwal

Reputation: 3860

AWS S3 will return you Forbidden(403) even if file does not exist for security reasons. Please ensure you have given proper s3 path while downloading.

You can read more about it here

Upvotes: 0

Related Questions