CODEWITHSUNDEEP
amazon-web-servicesaws-cloudformationaws-glue
Ernest Sadykov
Ernest Sadykov

Reputation: 831

How to get Glue development endpoint URL inside CloudFormation template?

I'm trying to write a CloudFormation template that creates:

The second resource requires endpoint URL. AWS::Glue::DevEndpoint provides endpoint name in the output, but it's not clear how to fetch the URL.

The only solution I found so far is to introduce aws glue get-dev-endpoint command and parse its output at some point of instance initialization process.

Upvotes: 6

Views: 916

Answers (1)

JD D
JD D

Reputation: 8137

As of May 2019, Cloudformation does not support retrieving the public address out of a AWS::Glue::DevEndpoint resource.

As you suggested in the question, you can get the dev endpoint name from CF and pass that to an AWS CLI command in the user data of an EC2 instance which is probably the best way to retrieve that.

I think the model error you are getting with the glue CLI may be resolved if you ensure you are using the latest version of the AWS CLI before you run the command. This error makes it sound like there is some sort of config issue with the glue service in the CLI.

Resources:
   MyDevEndpoint:
      Type: AWS::Glue::DevEndpoint
      Properties: 
         ...

   MyInstance:
      Type: AWS::EC2::Instance
      Properties:
         ...
         UserData:
            Fn::Base64: !Sub |
               aws --version
               yes | pip3 install awscli --upgrade --user
               aws --version
               ENDPOINT_NAME=${MyDevEndpoint}
               aws glue get-dev-endpoint --endpoint-name $ENDPOINT_NAME
               ...

Upvotes: 1

Related Questions