CODEWITHSUNDEEP
amazon-web-servicesaws-lambdaboto3amazon-emrautoscaling
Girish Jambkar
Girish Jambkar

Reputation: 13

AWS Create EMR using boto3 Autoscaling not working properly

I am auto scaling emr using boto3 and then autoscaling it using EMR_AutoScaling_DefaultRole. Autoscaling not working properly:

Initially it gives a warning as

The policy is pending attachment.

and at last failing giving error as

The Auto Scaling policy for instance group ig-XXXXXXXXXXXX in Amazon EMR cluster j-XXXXXXXXXXXX (test_emr...) could not attach and failed at 2018-12-19 10:03 UTC.

What can I do to assign the Autoscaling role?

Upvotes: 0

Views: 1298

Answers (2)

Ritesh
Ritesh

Reputation: 1034

In addition to answer by @Harsh Bafna. You also need to set Trust Relationship policy for EMR_AutoScaling_DefaultRole role.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Service": [
          "application-autoscaling.amazonaws.com",
          "elasticmapreduce.amazonaws.com"
        ]
      },
      "Action": "sts:AssumeRole"
    }
  ]
}

Upvotes: 4

Harsh Bafna
Harsh Bafna

Reputation: 2224

You need to add the AmazonElasticMapReduceforAutoScalingRole policy to the EMR_AutoScaling_DefaultRole.

Go to "IAM > Roles > EMR_AutoScaling_DefaultRole" and in the permissions tab click attach and add the AmazonElasticMapReduceforAutoScalingRole.

Also, to troubleshoot these failures, you can use the describe-cluster command from CLI. The response will include the Auto Scaling policy status, which gives an error message for failure reason if the policy has failed to attach.

Upvotes: 0

Related Questions