CODEWITHSUNDEEP
amazon-web-servicesamazon-s3jupyter-notebookamazon-sagemaker
Mike Jensen
Mike Jensen

Reputation: 3

AWS SageMaker S3 os.listdir() Access denied

I'm Stumped.

I took my TensorFlow model and moved it up into SageMaker to try it out. I put my own data up into an s3 bucket, set all the IAM roles/access (or so I think). I can read a file from s3. I can push a new file to s3. I can read local directories from my SageMaker local directories.

I cannot traverse my s3 bucket directories. I turned on logging and I get AccessDenied messages whenever I try access a URI of this format 's3://my_bucketName_here/Directory_of_my_data/'.

Here is what I've done: I've confirmed that my notebook uses the AmazonSageMaker-ExecutionRole-*** I've added AmazonSageMakerFullAccess Policy to that default role I've subsequently added AmazonS3FullAccess Policy as well

I then created a bucket policy specifically granting s3:* access on the specific bucket to that specific role.

Heck, I eventually made the bucket public with ListObjects = Yes.

os.listdir() simply fails with file or directory not found and a lot message is created with AccessDenied. (TensorFlow libraries just didn't work, so I went with os.listdir() to simplify things.

Finally, I test my access from the Policy Simulator - I selected the Role mentioned above, selected to test s3 and selected all 69 items and they all passed.

But I continue to log AccessDenied and cannot actually list the contents of a directory from my SageMaker jupyter notebook.

I'm at a loss. Thoughts?

EDIT: Per suggestion below, I have the following: bucket name contains sagemaker: '[redacted]-test-sagemaker' Public access is off, and the only account is my root account. { "Version": "2012-10-17", "Statement": [ { "Action": [ "s3:ListBucket" ], "Effect": "Allow", "Resource": [ "arn:aws:s3:::[redacted]-test-sagemaker" ] }, { "Action": [ "s3:GetObject", "s3:PutObject", "s3:DeleteObject" ], "Effect": "Allow", "Resource": [ "arn:aws:s3:::[redacted]-test-sagemaker/*" ] } ] } and arn:aws:iam::aws:policy/AmazonSageMakerFullAccess

Finally the bucket policy after the above failed: { "Id": "Policy1534116031672", "Version": "2012-10-17", "Statement": [ { "Sid": "Stmt1534116026409", "Action": "s3:*", "Effect": "Allow", "Resource": "arn:aws:s3:::[redacted]-test-sagemaker", "Principal": { "AWS": [ "arn:aws:iam::[id]:role/service-role/AmazonSageMaker-ExecutionRole-***" ] } } ] }

Upvotes: 0

Views: 3521

Answers (1)

Gili Nachum
Gili Nachum

Reputation: 5568

So you need to troubleshoot. Here are a few things to check:

0) Make sure the bucket is in the SageMaker region.

1) Include the string "sagemaker" in your bucket name (e.g., my_bucketName_here-sagemaker, SageMaker has out of the box access to buckets named this way.

2) Try using the SageMaker S3 default_bucket():

import sagemaker
s = sagemaker.Session()
s.upload_data(path='somefile.csv', bucket=s.default_bucket(), key_prefix='data/train')

3) Open terminal on the Notebook instance, to try to list your bucket using AWS CLI in bash:

aws iam get-user
aws s3 ls my_bucketName_here

Finally, pasting the bucket's access and resource policy in your question could help others to answer you.

Upvotes: 1

Related Questions