CODEWITHSUNDEEP
amazon-web-servicesamazon-data-pipeline
goelakash
goelakash

Reputation: 2519

Set 'maxActiveInstances' error

I am using AWS data-pipeline to export a DDB table, but when I activate I get an error:

Web service limit exceeded: Exceeded number of concurrent executions. Please set the field 'maxActiveInstances' to a higher value in your pipeline or wait for the currenly running executions to complete before trying again (Service: DataPipeline; Status Code: 400; Error Code: InvalidRequestException; Request ID: efbf9847-49fb-11e8-abef-1da37c3550b5)

How do I set this maxActiveInstances property using the AWS UI?

Upvotes: 2

Views: 3013

Answers (3)

ellcub
ellcub

Reputation: 601

With an on-demand pipline you can specify it in the 'Default object', like this

{
  "objects": [
    {
      "failureAndRerunMode": "CASCADE",
      "scheduleType": "ONDEMAND",
      "name": "Default",
      "id": "Default",
      "maxActiveInstances": "5"
    },
...

I couldn't add it in Architect, I had to create another pipeline from the json. But once that was done I could edit it in Architect (under 'Others' section).

Upvotes: 0

Josh G
Josh G

Reputation: 53

We ran into this too. For an on-demand pipeline, it looks like after a certain number of retries, you have to give it time to finish terminating the provisioned resources before you will be allowed to try again.

Solution: Patience.

Upvotes: 3

bruckerrlb
bruckerrlb

Reputation: 305

You can set it as a property on your Ec2Resource[1] (or EmrActivity[2]) object. Using the UI, click Edit Pipeline, click on Resources on the right hand side of the screen (it's a collapsable menu). There should be an Ec2Resource object. There should be a drop down on this object called "Add an additional field" and you should see max active instances in the drop down.

[1]https://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-object-ec2resource.html [2] https://docs.aws.amazon.com/datapipeline/latest/DeveloperGuide/dp-object-emractivity.html

Upvotes: 3

Related Questions