CODEWITHSUNDEEP
apache-sparkamazon-s3amazon-emrapache-hudi
Albert T. Wong
Albert T. Wong

Reputation: 1653

Running Spark Shell with Hudi, S3, EMR. Getting not authorized to perform: glue:GetDatabase on resource

the glue database "default" has already been created.

scala> (inputDF.write
     |     .format("hudi")
     |     .options(hudiOptions)
     |     .option(DataSourceWriteOptions.OPERATION_OPT_KEY,"insert")
     |     .mode(SaveMode.Overwrite)
     |     .save("s3a://oh-project-sunnyvalelabs-observed/marketing/"))
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
24/08/13 20:40:34 WARN HoodieSparkSqlWriterInternal: hoodie table at s3a://oh-project-sunnyvalelabs-observed/marketing already exists. Deleting existing data & overwriting with new data.
24/08/13 20:40:36 WARN HiveConf: HiveConf of name hive.server2.thrift.url does not exist
24/08/13 20:40:36 ERROR AWSCatalogMetastoreClient: Unable to verify existence of default database:
com.amazonaws.services.glue.model.AccessDeniedException: User: arn:aws:sts::654654235321:assumed-role/AmazonEMR-InstanceProfile-20240813T103425/i-0bbd3af62642a1490 is not authorized to perform: glue:GetDatabase on resource: arn:aws:glue:us-west-2:654654235321:catalog because no identity-based policy allows the glue:GetDatabase action (Service: AWSGlue; Status Code: 400; Error Code: AccessDeniedException; Request ID: 83a34deb-a334-4e38-96bb-14ff195f525b; Proxy: null)

Upvotes: 0

Views: 32

Answers (1)

Albert T. Wong
Albert T. Wong

Reputation: 1653

Need to modify AWS IAM Role AmazonEMR-InstanceProfile-20240813T103425 to include AWSGlueServiceRole

Upvotes: 0

Related Questions