CODEWITHSUNDEEP
amazon-s3google-cloud-storage
Jaya Ananthram
Jaya Ananthram

Reputation: 3463

Google storage transfer job (From Amazon s3 bucket) fails with error Unauthorized

I had created a google storage transfer job which has the following configuration

enter image description here

Source S3 bucket has the following policy,

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::jaya/*",
                "arn:aws:s3:::jaya"
            ]
        }
    ]
}

The transfer job kicks but failed for an unknown reason, the reason I got from the transfer job is,

Object: s3_fetch:0001_part_00.gz
Details: Http error code: Unauthorized.

The funniest thing is, source bucket has the file 0001_part_00.gz which means transfer job can able to fetch the file name from s3 bucket but it couldn't able to download from s3. What could be the reason?

Upvotes: 1

Views: 741

Answers (1)

cgranier
cgranier

Reputation: 130

I was having the same issue all day today but now it's working.

I tried creating a bucket from within the job creation process and it worked. I then recreated one of my previously failing jobs (to a bucket created outside the transfer job) and it's also working.

So maybe something was broken at AWS or GCP today.

Upvotes: 1

Related Questions