CODEWITHSUNDEEP
aws-step-functions
Adrian Hoffman
Adrian Hoffman

Reputation: 321

AWS Step Function Task Token Prior To Wait

Good Morning Everyone,

I am trying to manage some async actions via the step function but I need to be able to send and receive the token during separate phases. I have been looking at the documentation and it seems like this isn't possible as the await token is only a specific task within a step function and can't be created ahead of time.

Is this setup possible or do I need to restructure to accommodate the restriction of a task token is specific to a task (ie. I would have to await on the send of the data even tho that isn't the process that will receive the data)?

enter image description here

Upvotes: 1

Views: 2054

Answers (1)

fedonev
fedonev

Reputation: 25639

As you say, "Callback tasks provide a way to pause a workflow until a task token is returned." But you don't want to wait. If I understand right, you want to fire off the job to the 3rd Party do some more work in your Step function, and only then wait for the processed results.

How about this orchestration, which does not make use of tokens:

Lambda-1 (SFN) -> 3rd Party -> Lambda-2 -> SQS Queue: Lambda-1 sends a request to the 3rd Party, which completes its work and it passes the response to Lamba-2. Lambda-2 puts its result into a Queue. The only change here is SQS at the end.

You Step Function looks like:

  1. Lambda-1 sends data to 3rd Party - no .callback, just fire off the request and return
  2. (Do more work in the Step Function)
  3. Poll the Queue for a Result, using the newly introduced AWS SDK service integrations.
  4. If no result, wait and repeat up to 10 minutes, periodically repolling the queue.
  5. Lambda-3 and Lambda-4 if success or failure

Using a queue in this way robustly lets either branch (Lambda-2 or More-Work) to finish earlier.

Upvotes: 1

Related Questions