CODEWITHSUNDEEP
amazon-web-servicesboto3aws-step-functions
Hari Kaushik
Hari Kaushik

Reputation: 61

In an AWS Step Function's Parallel state, if one branch fails is there a way to ensure that the other branches continue until completion?

I have a step function with a parallel state that has four branches (A, B, C, D) each with 3 Glue StartJobRun task states. In the case that branch A fails during the 1st task, how can I make sure the other branches continue with their tasks as long as they are successful?

Upvotes: 6

Views: 7849

Answers (2)

Rishank Verma_CS0136
Rishank Verma_CS0136

Reputation: 1

you can make trigger your glue crawler from the any async mechanism. Once crawler completes it job make sure to provide success to step function.

Upvotes: 0

Pooya Paridel
Pooya Paridel

Reputation: 1401

As you mentioned in AWS Step Functions if one branch fails, the other branches will be immediately terminated and their results ignored. Step Functions will make a best-effort attempt to cancel any .sync tasks that are in progress, and stop executing any further states on any branch. If you want to let those other branches proceed to completion, you have to prevent any branch from failing. You can use Catch blocks on each state and transition to a Succeed (even when it was actually a failure). Then, after the Parallel completes, add a new state to do error handling to check which branches succeeded or failed.

Upvotes: 3

Related Questions