How to use jsonPath inside array in AWS Step Functions

Question

I am writing an AWS step function, and for one of the steps, I wish to call a lambda that accepts an array as one of the inputs. However, if I try to pass in a JsonPath into the array, I get

The value for the field 'arrayField.$' must be a STRING that contains a JSONPath but was an ARRAY

My step function definition:

{
  "StartAt": "First",
  "States": {
    "First": {
      "Type": "Pass",
      "Parameters": {
        "type": "person"
      },
      "ResultPath": "$.output",
      "Next": "Second"
    },
    "Second": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:<aws_id>:function:MyFunction",
      "Parameters": {
        "regularParameter": "some string",
        "arrayParameter.$": ["$.output.type"]
      },
      "Next": "Succeed"
    },
    "Succeed": {
      "Type": "Succeed"
    }
  }
}

How can I use jsonPath inside the array?

Answer 1

Since a new release you could use the intrinsic function States.Array:

  "arrayParameter.$": "States.Array($.output.type)"

https://docs.aws.amazon.com/step-functions/latest/dg/amazon-states-language-intrinsic-functions.html

Answer 2

Another way to approach this is by using Parallel state that outputs an array of objects and then use jsonPath to convert it to a simple array:

{
  "StartAt": "Parallel",
  "States": {
    "Parallel": {
      "Type": "Parallel",
      "Next": "Use Array",
      "ResultPath": "$.items",
      "Branches": [
        {
          "StartAt": "CreateArray",
          "States": {
            "CreateArray": {
              "Type": "Pass",
              "Parameters": {
                "value": "your value"
              },
              "End": true
            }
          }
        }
      ]
    },
    "Use Array": {
      "Type": "Pass",
      "Parameters": {
        "items.$": "$.items[*].value"
      },
      "End": true
    }
  }
}

In this example, Parallel state outputs the following json:

{
  "items": [
    {
      "value": "your value"
    }
  ]
}

And "Use Array" state produces:

{
  "items": [
    "your value"
  ]
}

Answer 3

As many answers correctly pointed out, it's not possible to do it exactly the way you need. But I would suggest another solution: an array of dictionaries. It's not exactly what you need, but is native and not hacky.

"Second": {
  "Type": "Task",
  "Resource": "arn:aws:lambda:us-east-1:<aws_id>:function:MyFunction",
  "Parameters": {
    "regularParameter": "some string",
    "arrayParameter": [{"type.$": "$.output.type"}]
  },
  "Next": "Succeed"
},

The result would be

{
  "regularParameter": "some string",
  "arrayParameter": [{"type": "SingleItemWrappedToAnArray"}]
}

Answer 4

As @Seth Miller mentioned JsonPath resolution within arrays doesn't work unfortunately. If the amount of values to replace in the array is small and known there's a simple workaround (in my case I needed an array of size 1).

The steps are:

1. Initialise the array with the number of values you need;
2. Replace each value using "ResultPath": "$.path.to.array[n]";
3. Use "$.path.to.array" in your task.

Simple, working example:

{
  "StartAt": "First",
  "States": {
    "First": {
      "Type": "Pass",
      "Parameters": {
        "type": "person"
      },
      "ResultPath": "$.output",
      "Next": "Initialise Array"
    },
    "Initialise Array": {
      "Comment": "Add an entry for each value you intend to have in the final array, the values here don't matter.",
      "Type": "Pass",
      "Parameters": [
        0
      ],
      "ResultPath": "$.arrayParameter",
      "Next": "Fill Array"
    },
    "Fill Array": {
      "Comment": "Replace the first entry of array with parameter",
      "Type": "Pass",
      "InputPath": "$.output.type",
      "ResultPath": "$.arrayParameter[0]",
      "End": true
    }
  }
}

And to use the resulting array in your task example:

    "Second": {
      "Type": "Task",
      "Resource": "arn:aws:lambda:us-east-1:<aws_id>:function:MyFunction",
      "Parameters": {
        "regularParameter": "some string",
        "arrayParameter.$": "$.arrayParameter"
      },
      "Next": "Succeed"
    },

Answer 5

Reasonable advice

I ran into a use case for JsonPath resolution within arrays today and found (like you have) that the functionality does not exist today. I ended up deciding that doing the data massaging in code was simpler and cleaner. For example, you could create a small Lambda that takes in the object emitted by First and massages it to a format acceptable to Second and adds it to the output (WaterKnight mentions this solution in a comment to another question).

This assumes that you are, for some reason, unable to change the format of the input to that Lambda in Second (which would be the absolute shortest path here).

Unreasonable advice

That said, if you want a way to do this completely within Step Functions that is fairly gross, you can use the result of a Map state that executes Pass states. The output of the Map state is an array that aggregate the output of each constituent Pass state. These Pass states simply emit the value(s) you want in the final array using the Parameters attribute. An example Step Function definition follows. I did warn that it is gross and that I went a different way to solve the problem.

 {
    "StartAt": "First",
    "Comment": "Please don't actually do this",
    "States": {
      "First": {
        "Type": "Pass",
        "Parameters": {
          "type": "person"
        },
        "ResultPath": "$.output",
        "Next": "Add Array"
      },
      "Add Array": {
        "Comment": "A Map state needs some array to loop over in order to work. We will give it a dummy array. Add an entry for each value you intend to have in the final array. The values here don't matter.",
        "Type": "Pass",
        "Result": [
          0
        ],
        "ResultPath": "$.dummy",
        "Next": "Mapper"
      },
      "Mapper": {
        "Comment": "Add a Pass state with the appropriate Parameters for each field you want to map into the output array",
        "Type": "Map",
        "InputPath": "$",
        "ItemsPath": "$.dummy",
        "Parameters": {
          "output.$": "$.output"
        },
        "Iterator": {
          "StartAt": "Massage",
          "States": {
            "Massage": {
              "Type": "Pass",
              "Parameters": {
                "type.$": "$.output.type"
              },
              "OutputPath": "$.type",
              "End": true
            }
          }
        },
        "ResultPath": "$.output.typeArray",
        "Next": "Second"
      },
      "Second": {
        "Comment": "The Lambda in your example is replaced with Pass so that I could test this",
        "Type": "Pass",
        "Parameters": {
          "regularParameter": "some string",
          "arrayParameter.$": "$.output.typeArray"
        },
        "Next": "Succeed"
      },
      "Succeed": {
        "Type": "Succeed"
      }
    }
  }

Answer 6

JSONPath inside parameters field need to be a string. So if you want to pass to lambda function a parameter called arrayParameter, you´ll need to make a jsonPath query that extract that array.

For example, if inside the key output is a key called outputArray with the array as its value.

Input JSON:

{
  "pre": "sdigf",
  "output": {
    "result": 1,
    "outputArray": ["test1","test2","test.."]
  }
}

The parameter sintax:

"arrayParameter.$": "$.output.outputArray"