Unable to configure SageMaker execution Role with access to S3 bucket in another AWS account

Question

Requirement: Create SakeMaker GroundTruth labeling job with input/output location pointing to S3 bucket in another AWS account

High Level Steps Followed: Lets say, Account_A: SageMaker GroundTruth labeling job and Account_B: S3 bucket

1. Create role AmazonSageMaker-ExecutionRole in Account_A with 3 policies attached:

• AmazonSageMakerFullAccess
• Account_B_S3_AccessPolicy: Policy with necessary S3 permissions to access S3 bucket in Account_B
• AssumeRolePolicy: Assume role policy for arn:aws:iam::Account_B:role/Cross-Account-S3-Access-Role

2. Create role Cross-Account-S3-Access-Role in Account_B with 1 policy and 1 trust relationship attached:

• S3_AccessPolicy: Policy with necessary S3 permissions to access S3 bucket in the this Account_B
• TrustRelationship: For principal arn:aws:iam::Account_A:role/AmazonSageMaker-ExecutionRole

Error: While trying to create SakeMaker GroundTruth labeling job with IAM role as AmazonSageMaker-ExecutionRole, it throws error AccessDenied: Access Denied - The S3 bucket 'Account_B_S3_bucket_name' you entered in Input dataset location cannot be reached. Either the bucket does not exist, or you do not have permission to access it. If the bucket does not exist, update Input dataset location with a new S3 URI. If the bucket exists, give the IAM entity you are using to create this labeling job permission to read and write to this S3 bucket, and try your request again.

Answer 1

Reverted back to original approach where access to the SageMaker execution role was provided through direct S3 bucket policy.

While creating the GT job from console:

(i) Expects the user creating the job also to have access to the data in cross account S3 bucket; Updated bucket policy to have access for both SageMaker execution role as well as user

(ii) Expects the manifest in own account's S3 bucket; Fails with 403 if manifest is in cross account S3 bucket even though SageMaker execution role had access to the cross account S3 bucket

While creating the GT job from CLI: Above restrictions doesn't apply and was able to create the GT job.

Answer 2

In your high level step 2, the approach should change to using a Resource Policy on your S3 bucket that allows account A to write to it. Rather than expecting Account A to assume a role in Account B, which I don't believe Sagemeker will do. Therefore the general approach is to do the following:

1. Account A Sagemaker service is given has a iam policy with a that allows access to Account B bucket. (Basically what you've done).

2. Account B bucket is given a resource policy that allows Account A to access it.

The following article gives additional help on this topic: How can I provide cross-account access to objects that are in Amazon S3 buckets?