CODEWITHSUNDEEP
amazon-web-servicesamazon-emr
pikinier20
pikinier20

Reputation: 1

Access denied on table access to AWS Lake Formation using AWS EMR

I'm working on a small pet project in which I want to setup AWS Lake Formation, put there some data and then access this data in AWS EMR. I was following instructions in this doc: https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-lake-formation.html. I've already got:

  1. A data lake with some CSVs
  2. Created table schemas for these CSVs
  3. A external IdP (Auth0)
  4. IAM Roles as in the tutorial.
  5. The EMR cluster set up as in tutorial. The problem is that I cannot access data in data lake. I can list databases and tables but can't perform a SELECT on any table. I get:
com.amazonaws.emr.recordserver.remote.RecordServerException: com.amazonaws.emr.recordserver.remote.RecordServerException: Error while trying to get temporary table credentials for user: 5000, table: ***, database: ***, catalog: null from Secret Agent.
...
Caused by: SecretAgentClientException{httpResponseCode=Optional[500] message=Access denied for tableArn=arn:aws:glue:eu-west-1:743262912284:TABLE_NAME userId=5000 permissions=SELECT cause.class=null}

I checked the logs in Lake Formation and there I've got a DataAccess event with principal set to the user that I use in Zeppelin notebook.

{
    "eventVersion": "1.08",
    "userIdentity": {
        "type": "AssumedRole",
        "principalId": "***",
        "arn": "***",
        "accountId": "***",
        "accessKeyId": "***",
        "sessionContext": {
            "sessionIssuer": {
                "type": "Role",
                "principalId": "***",
                "arn": "PROPER_ROLE_ARN",
                "accountId": "***",
                "userName": "PROPER_ROLE"
            },
            "attributes": {
                "creationDate": "2022-01-14T10:26:39Z",
                "mfaAuthenticated": "false"
            }
        }
    },
    "eventTime": "2022-01-14T10:53:40Z",
    "eventSource": "lakeformation.amazonaws.com",
    "eventName": "GetDataAccess",
    "awsRegion": "eu-west-1",
    "sourceIPAddress": "***",
    "userAgent": "aws-sdk-java/1.11.970 Linux/4.14.252-195.483.amzn2.x86_64 OpenJDK_64-Bit_Server_VM/25.312-b07 java/1.8.0_312 vendor/Amazon.com_Inc.",
    "errorCode": "AccessDenied",
    "errorMessage": "Access is not allowed.",
    "requestParameters": {
        "tableArn": "arn:aws:glue:eu-west-1:***:***",
        "permissions": [
            "SELECT"
        ],
        "durationSeconds": 3600
    },
    "responseElements": null,
    "additionalEventData": {
        "requesterService": "UNKNOWN",
        "LakeFormationTrustedCallerInvocation": "true",
        "lakeFormationPrincipal": "PROPER_PRINCIPAL_ARN"
    },
    "requestID": "1e4c02e9-e09e-4906-a96f-c29a882b46de",
    "eventID": "b5b26de0-60f4-482c-b670-7fba9d7089e5",
    "readOnly": true,
    "eventType": "AwsApiCall",
    "managementEvent": true,
    "recipientAccountId": "***",
    "eventCategory": "Management",
    "tlsDetails": {
        "clientProvidedHostHeader": "lakeformation.eu-west-1.amazonaws.com"
    }
}

The assumed role in sessionContext is correct. I've checked policies of created roles multiple times and they are exactly as in the tutorial (they have granted access to the lakeformation and to glue and also to all tables)

Upvotes: 0

Views: 2205

Answers (2)

spieden
spieden

Reputation: 1

I encountered this same error message and similarly double checked everything. In my case the issue was that the default service-linked role didn't include needed KMS policy for the key on the relevant bucket. After registering the bucket with Lake Formation using a custom role that included the same S3 actions allowed by the service-linked role plus "kms:Encrypt", "kms:Decrypt", "kms:ReEncrypt*", "kms:GenerateDataKey*", "kms:DescribeKey" on the relevant KMS key I was GTG.

More info here: https://docs.aws.amazon.com/lake-formation/latest/dg/register-encrypted.html

Upvotes: 0

Austin Lee
Austin Lee

Reputation: 84

Please ensure that you allow each account to run Lake Formation-enabled EMR clusters. You can do so on the Lake Formation console by going to "External Data Filtering" found on the left navigation panel.

https://docs.aws.amazon.com/lake-formation/latest/dg/getting-started-setup.html#emr-switch.

Upvotes: 0

Related Questions