CODEWITHSUNDEEP
amazon-web-servicesterraformterraform-provider-awsamazon-sagemakeraws-step-functions
wawawa
wawawa

Reputation: 3355

How to append stepfunction execution id to SageMaker job names?

I have a step function statemachine which creates SageMaker batch transform job, the definition is written in Terraform, I wanted to add the stepfunction execution id to the batch transform job names:

in stepfunction terraform file:

  definition = templatefile("stepfuntion.json",
    {
      xxxx
)

in the "stepfuntion.json":

{...
          "TransformJobName": "jobname-$$.Execution.Id",
  
          }
      },
        "End": true
      }
    }
  }

But after terraform apply, it didn't generate the actual id, it gave me jobname-$$.Execution.Id, can anyone help with this please?

Resources: https://docs.aws.amazon.com/step-functions/latest/dg/input-output-contextobject.html "To access the context object, first specify the parameter name by appending .$ to the end, as you do when selecting state input with a path. Then, to access context object data instead of the input, prepend the path with $$.. This tells AWS Step Functions to use the path to select a node in the context object."

Can someone tell me what I'm missing please?

Upvotes: 0

Views: 1427

Answers (2)

Junior
Junior

Reputation: 11

If I'm reading wawawa right, we are almost there. Terraform creates the function, not an execution instance of it. The Stepfunction assigns an execution ID only when it executes - and it changes every time! So let Terraform create the code with a variable that Step Function will resolve at execution time.

So TransformJobName is getting the string literal "jobname--$$Execution.Id". The right side looks fine, but the instruction the OP cited is trying to say we need to append a ".$" to your parameter name (on the left) to activate that substitution. The instruction is written poorly but their examples are correct.

so your pair should look like this:

"TransformJobName.$": "jobname-$$.Execution.Id",

I've found almost always my assignments say x.$ = $.y to reach into input or output, or x.$ = $$.y if it's a context object.

Bonus tip: this is how I send arguments to Glue tasks. Context is NOT part of the input path so you have to do something to include the context you want (execution.id in my case) as arguments.

    "Arguments": {
      "--inputbucket.$": "$.detail.bucket.name",
      "--inputobject.$": "$.detail.object.key",
      "--executionid.$": "$$.Execution.Id"
    }

Upvotes: 1

samtoddler
samtoddler

Reputation: 9665

The var you are trying to use terraform doesn't know about it

jobname-$$.Execution.Id.

That's something specific to the Step function and available within state machine not available for terraform.

Upvotes: 1

Related Questions