CODEWITHSUNDEEP
amazon-web-servicessecurityamazon-iam
Istvan
Istvan

Reputation: 8572

How to determine which particular permission is missing for using a particular AWS feature?

I regularly run into issues that one particular action is missing from the IAM user using AWS. A good example is uploading data to S3. Hadoop throws the following exception:

2018-08-03 09:29:46,112 INFO [IPC Server handler 27 on 42415] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Progress of TaskAttempt attempt_1526322305732_0008_m_000008_0 is : 0.0
2018-08-03 09:29:46,134 FATAL [IPC Server handler 26 on 42415] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: attempt_1526322305732_0008_m_000008_0 - exited : com.cloudera.com.amazonaws.services.s3.model.AmazonS3Exception: Forbidden (Service: Amazon S3; Status Code: 403; Error Code: 403 Forbidden; Request ID: 008CAB66479B6842), S3 Extended Request ID: JtUlF07hBNr03NhytAQj6biGX8I/YKjtbUcz82PkjbLoDeoW3W8AVLvhAdXWk7V9Fc8G4oOy1d8=
    at com.cloudera.com.amazonaws.http.AmazonHttpClient.handleErrorResponse(AmazonHttpClient.java:1182)
    at com.cloudera.com.amazonaws.http.AmazonHttpClient.executeOneRequest(AmazonHttpClient.java:770)
    at com.cloudera.com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:489)
    at com.cloudera.com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:310)
    at com.cloudera.com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:3785)
    at com.cloudera.com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1050)
    at com.cloudera.com.amazonaws.services.s3.AmazonS3Client.getObjectMetadata(AmazonS3Client.java:1027)
    at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:961)
    at org.apache.hadoop.fs.s3a.S3AFileSystem.getFileStatus(S3AFileSystem.java:78)
    at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1412)
    at org.apache.hadoop.tools.mapred.CopyMapper.setup(CopyMapper.java:114)
    at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)
    at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
    at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
    at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
    at java.security.AccessController.doPrivileged(Native Method)
    at javax.security.auth.Subject.doAs(Subject.java:415)
    at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1693)
    at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)

This exception does not have enough information to determine which action is missing for the user. What is the best way of identifying which action must be added for such a task?

Upvotes: 1

Views: 397

Answers (1)

Nune Isabekyan
Nune Isabekyan

Reputation: 549

We frequently use CloudTrail, where you can see what was the attempted call and if the response was successful or not, and what was the reason.

Upvotes: 2

Related Questions