CODEWITHSUNDEEP
amazon-web-servicesamazon-emr
wrschneider
wrschneider

Reputation: 18770

AWS EMR with Glue catalog, specify catalogId explicitly

is there a way to specify a Glue catalogId explicitly in EMR configuration?

https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-hive-metastore-glue.html

This shows how to specify to use Glue in EMR for Hive metastore, but does not document any parameter to specify the catalogId that would be passed to Glue API calls (presumably, defaults to the account ID).

Later on the doc does mention "... if the cluster is in a different AWS account" which seems to imply there is a way to point the EMR cluster at a Glue catalog in a different account.

Upvotes: 2

Views: 2648

Answers (1)

wrschneider
wrschneider

Reputation: 18770

It looks like this was recently added. The relevant property is hive.metastore.glue.catalogid, within hive-site

https://docs.aws.amazon.com/emr/latest/ReleaseGuide/emr-hive-metastore-glue.html

"In addition, with Amazon EMR 5.16.0 and later, you can use the configuration classification to specify a Data Catalog in a different AWS account."

"To specify a Data Catalog in a different AWS account, add the hive.metastore.glue.catalogid property as shown in the following example"

Upvotes: 3

Related Questions