CODEWITHSUNDEEP
amazon-redshiftaws-lake-formationaws-iam-identity-center
Indoordinosaur
Indoordinosaur

Reputation: 127

AWS Lake Formation/Redshift Integration with IAM IDC getting permission denied with everything except IAMAllowedPrincipals

I've been going down the AWS Lake Formation (LF) rabbit hole trying for days to get it to work with Redshift and IAM Identity Center (IAM IDC).

The intended goal:

Before messing around with the new Lake Formation we had a functional data lake that can be queried in Redshift QueryEditorV2 using the normal federated users and JDBC user connections, the connections still work. I have managed to get Redshift integrated with our IDC and have a test user who is part of a group added to an IDC group which is also available to be assigned to tables and resources in AWS Lake Formation. When I log into the QueryEditorV2 with that user I was initially able to query tables which were made visible already before the LF work.

However, I've realized that I am only able to query the tables successfully when the table has LF permissions via the IAMAllowedPrincipals group. When this is revoked and I grant the permissions via either the IAM IDC user directly or the IAM IDC Group which the user belongs to I am unable to query. I get the below error message in QueryEditorV2:

ERROR: AwsClientException: AccessDeniedException from glue - Insufficient Lake Formation permission(s) on example_table[ErrorId: 1-66a178f2-00168194599db4c26280f923]

In short, it seems that my Redshift IDC user is authenticating with LF via the IAMAllowedPrincipals pathway to access data rather than Identity Center credentials. I won't list all the things I've tried here but I've spent a couple days going through the two tutorials I listed above as well as all other Lake Formation documentation which seemed useful. I've tried several combinations of LF permission settings, data location registration, hybrid access vs. no hybrid access and am just having a hard time understanding why the only permission set that seems to work for anything are the IAM roles which would have had access to the data lake anyways.

Upvotes: 0

Views: 236

Answers (1)

Indoordinosaur
Indoordinosaur

Reputation: 127

Finally figured this out after talking with AWS Support. Turns out that this is caused by a known bug regarding partitions on tables https://docs.aws.amazon.com/lake-formation/latest/dg/limitations.html#issue-table-partitions.

Unfortunately, it seems like this issue has been outstanding for years.

Upvotes: 0

Related Questions