CODEWITHSUNDEEP
amazon-web-servicesaws-step-functions
hatellla
hatellla

Reputation: 5142

Parallel States Merge the output in Step Function

Is it possible to have following kind of Step Function graph, i.e. from 2 parallel state output, one combined state:

enter image description here

If yes, what would json for this looks like? If not, why?

Upvotes: 24

Views: 31305

Answers (5)

tsunllly
tsunllly

Reputation: 1668

This worked for me

        "Transform And Freeze": {
          "Type": "Parallel",
          "InputPath": "$",
          "Branches": [
            {
              "StartAt": "Transform Status",
              "States": {
                "Transform Status": {
                  "Type": "Map",
                  "ItemsPath": "$",
                  "MaxConcurrency": 25,
                  "Iterator": {
                    "StartAt": "Transform",
                    "States": {
                      "Transform": {
                        "Type": "Task",
                        "Resource": "${TransformFunction}",
                        "End": true
                      }
                    }
                  },
                  "End": true
                }
              }
            },
            {
              "StartAt": "Freeze Status",
              "States": {
                "Freeze Status": {
                  "Type": "Map",
                  "MaxConcurrency": 25,
                  "Iterator": {
                    "StartAt": "Freeze",
                    "States": {
                      "Freeze Transactions": {
                        "Type": "Task",
                        "Resource": "${FreezeFunction}",
                        "End": true
                      }
                    }
                  },
                  "End": true
                }
              }
            }
          ],
          "ResultPath" : "$.parts",
          "Next": "SetParallelOutput",
          "Catch": [
            {
              "ErrorEquals": [
                "States.ALL"
              ],
              "ResultPath": "$.exception",
              "Next": "Error Handler"
            }
          ]
        },
        "SetParallelOutput": {
          "Type": "Pass",
          "Parameters": {
            "foo.$": "$.foo",
            "bar.$": "$.bar",
            "parts.$": "$.parts[0]"
          },
          "Next": "Target Type"
        },

enter image description here

Upvotes: 0

Austin Han Wang
Austin Han Wang

Reputation: 59

We can use ResultSelector and Result Path to combine the result into one object

We have a parallel state like:

{
  "StartAt": "ParallelBranch",
  "States": {
    "ParallelBranch": {
      "Type": "Parallel",
      "ResultPath": "$",
      "InputPath": "$",
      "OutputPath": "$",
      "ResultSelector": {
        "UsersResult.$": "$[1].UsersUpload",
        "CustomersResult.$": "$[0].customersDataUpload"
      },
      "Branches": [
        {
          "StartAt": "customersDataUpload",
          "States": {
            "customersDataUpload": {
              "Type": "Pass",
              "ResultPath": "$.customersDataUpload.Output",
              "Result": {
                "CompletionStatus": "success",
                "CompletionDetails": null
              },
              "Next": "Wait2"
            },
            "Wait2": {
              "Comment": "A Wait state delays the state machine from continuing for a specified time.",
              "Type": "Wait",
              "Seconds": 2,
              "End": true
            }
          }
        },
        {
          "StartAt": "UsersUpload",
          "States": {
            "UsersUpload": {
              "Type": "Pass",
              "Result": {
                "CompletionStatus": "success",
                "CompletionDetails": null
              },
              "ResultPath": "$.UsersUpload.Output",
              "Next": "Wait1"
            },
            "Wait1": {
              "Comment": "A Wait state delays the state machine from continuing for a specified time.",
              "Type": "Wait",
              "Seconds": 1,
              "End": true
            }
          }
        }
      ],
      "End": true
    }
  },
  "TimeoutSeconds": 129600,
  "Version": "1.0"
}

enter image description here

And the output will be like:

{
  "UsersResult": {
    "Output": {
      "CompletionStatus": "success",
      "CompletionDetails": null
    }
  },
  "CustomersResult": {
    "Output": {
      "CompletionStatus": "success",
      "CompletionDetails": null
    }
  }
}

Upvotes: 5

bounav
bounav

Reputation: 5046

A parallel task always outputs an array (containing one entry per branch).

You can tell AWS step functions to append the output into new (or existing) property in the original input with "ResultPath": "$.ParallelOut" in your parallel state definition, but this is not what you seem to be trying to achieve.

To merge the output of parallel task, you can leverage the "Type": "Pass" state to define transformations to apply to the JSON document.

For example, in the state machine below, I'm transforming a JSON array...

[
  {
    "One": 1,
    "Two": 2
  },
  {
    "Foo": "Bar",
    "Hello": "World"
  }
]

...into a few properties

{
  "Hello": "World",
  "One": 1,
  "Foo": "Bar",
  "Two": 2
}

Transform an array into properties with AWS Step Functions

{
    "Comment": "How to convert an array into properties",
    "StartAt": "warm-up",
    "States": {
      "warm-up": {
        "Type": "Parallel",
        "Next": "array-to-properties",
        "Branches": [
          {
            "StartAt": "numbers",
            "States": {
              "numbers": {
                "Type": "Pass",
                "Result": {
                    "One": 1,
                    "Two" : 2
                },
                "End": true
              }
            }
          },
          {
            "StartAt": "words",
            "States": {
              "words": {
                "Type": "Pass",
                "Result": {
                    "Foo": "Bar",
                    "Hello": "World"
                },
                "End": true
              }
            }
          }
        ]
      },
      "array-to-properties": {
        "Type": "Pass",
        "Parameters": {
          "One.$": "$[0].One",
          "Two.$": "$[0].Two",
          "Foo.$": "$[1].Foo",
          "Hello.$": "$[1].Hello"
        },
        "End": true
      }
    }
}

Upvotes: 23

Popoi Menenet
Popoi Menenet

Reputation: 1058

It is possible as opposed diagram below

enter image description here

The parallel state should look like this

"MyParallelState": {
  "Type": "Parallel",
  "InputPath": "$",
  "OutputPath": "$",
  "ResultPath": "$.ParallelResultPath",
  "Next": "SetCartCompleteStatusState",
  "Branches": [
    {
      "StartAt": "UpdateMonthlyUsageState",
      "States": {
        "UpdateMonthlyUsageState": {
          "Type": "Task",
          "InputPath": "$",
          "OutputPath": "$",
          "ResultPath": "$.UpdateMonthlyUsageResultPath",
          "Resource": "LambdaARN",
          "End": true
        }
      }
    },
    {
      "StartAt": "QueueTaxInvoiceState",
      "States": {
        "QueueTaxInvoiceState": {
          "Type": "Task",
          "InputPath": "$",
          "OutputPath": "$",
          "ResultPath": "$.QueueTaxInvoiceResultPath",
          "Resource": "LambdaARN",
          "End": true
        }
      }
    }

The output of MyParallelState will be populated as in array, from each state in the Parallel state. They are populated within ParallelResultPath object and will be passed into the Next state

{
  "ParallelResultPath": [
    {
      "UpdateMonthlyUsageResultPath": Some Output
    },
    {
      "QueueTaxInvoiceResultPath": Some Output
    }
  ]
}

Upvotes: 12

A.Khan
A.Khan

Reputation: 4002

Your diagram is technically wrong because no state can set multiple states to its Next task. You cannot start State Machine as StartAt by providing multiple State names. Also, even if it was possible I don't see any point why would you want to run two parallel states as opposed to one parallel state with all the sub states that you would split into two.

Upvotes: 1

Related Questions