Ktos
Ktos

Reputation: 75

AccessDeniedException when running DBT in AWS with Glue adapter: Insufficient Lake Formation permissions

I'm new to dbt and AWS, and I'm following a tutorial to learn how to use dbt with AWS Glue.

I ran into an issue when trying to run the following dbt command:

dbt run -m silver --profiles-dir profiles

I received this error:

Runtime Error in model silver_avg_metrics (models/silver/silver_avg_metrics.sql)
  An error occurred (AccessDeniedException) when calling the GetTable operation: Insufficient Lake Formation permission(s) on silver_avg_metrics

This error suggests that my AWS IAM role doesn't have the correct permissions in Lake Formation to access the table silver_avg_metrics. However, my understanding is that DBT is supposed to create this table automatically based on the SQL query in the model file, so I’m not sure why I'm receiving this error.

My IAM role has permissions for AWS Glue and Lake Formation, but I’m still seeing this error. (Actually it has admin permission for all resources).

Has anyone encountered this problem before? Could I be missing something in the configuration? Any advice or suggestions would be greatly appreciated.

TIA!

DBT version:

Core:
  - installed: 1.8.9
  - latest:    1.8.9 - Up to date!

Plugins:
  - glue:     1.8.6 - Up to date!
  - redshift: 1.8.1 - Up to date!
  - postgres: 1.8.2 - Up to date!
  - spark:    1.8.0 - Up to date!

Upvotes: 1

Views: 87

Answers (0)

Related Questions