CODEWITHSUNDEEP
amazon-web-servicesemrautoscalingaws-cloudformation
Jacek
Jacek

Reputation: 73

Cannot create AWS EMR with autoscaling via cloudformation

I am working on EMR template with autoscaling. While a static EMR setup with instance group works fine, I cannot attach

AWS::ApplicationAutoScaling::ScalableTarget

As a troubleshooting I've split my template into 2 separate ones. In first I am creating a normal EMR cluster (which is fine). And then in second I have a ScalableTarget definition which fails attach with error:

11:29:34 UTC+0100 CREATE_FAILED AWS::ApplicationAutoScaling::ScalableTarget AutoscalingTarget EMR instance group doesn't exist: Failed to find Cluster XXXXXXX

Funny thing is that this cluster DOES exist. I also had a look at IAM roles but everything seems to be ok there...

Can anyone advice on that matter? Did anyone for Autoscaling instancegroup to work via Cloudformation?

Upvotes: 3

Views: 842

Answers (1)

sagar
sagar

Reputation: 11

I have already tried and raised a request with AWS. This autoscaling feature is not yet available using CloudFormation. Now I am using CF for Custom EMR SecGrp creation and S3 etc and in output tab, I am adding Command line command(aws emr create-cluster...... ). After getting output querying the result to launch Cluster.

Actually, autoscaling can be enabled at the time of cluster launching by using --auto-scaling-role. If we use CF for EMR, autoscaling feature is not available because it launches cluster without "--auto-scaling-role".

I hope this can be useful...

Upvotes: 1

Related Questions