CODEWITHSUNDEEP
amazon-web-servicesamazon-emramazon-data-pipeline
Kristian
Kristian

Reputation: 21830

Order of EMR Steps

When initially setting up a new EMR cluster, and you add steps to your cluster, the order that they append to the page is top-down, meaning the top entry runs first, the bottom entry runs last.

When you are watching the progress of those steps in a running EMR cluster, they run bottom-up, meaning bottom runs first, top runs last.

When I export a CLI command for the setup of the cluster, and run that via DataPipeline, it contains the --steps flag which has an array of steps, the first-running step is last item in the array.

Needless to say, it is very confusing to me how I would actually mandate the order of particular steps for the ETL job I'm doing.

Is there a way to explicitly specify the order of these steps?

Upvotes: 2

Views: 2150

Answers (1)

Ryan Widmaier
Ryan Widmaier

Reputation: 8513

They do run in the order you specify them via the API, but for some reason the web console has weird sorting when displaying them.

Upvotes: 5

Related Questions