CODEWITHSUNDEEP
aws-step-functions
Stanley Kirdey
Stanley Kirdey

Reputation: 631

AWS Step Functions - multiple workers per same activity ARN

I can't seem to find good explanation of what would happen if several workers listen to the same AWS step function activity ARN? I mainly interested in the mechanics as I am exploring how to introduce fault tolerance to the processes that listen to step function activities, meaning if one worker fails, how another worker can pickup the slack from an activity task?

If there is a defined pattern - please share :-)

Thanks!

Upvotes: 1

Views: 1365

Answers (2)

Stanley Kirdey
Stanley Kirdey

Reputation: 631

During another conversation I think I've got the answer to this question:

In a case there are two workers in two different AZs that listen to the same activity ARN and one of the AZ goes down before worker in that AZ is able to get the taskToken, other worker can pick it up and start the work on the task.

In another case, if worker in the AZ1 picked the work and AZ1 goes down - the step will eventually timeout, but retry of the step could force worker in AZ2 to pick the work.

In a sense step function activity is a task queue.

Upvotes: 1

Ashan
Ashan

Reputation: 19728

In your worker implement the heart beat so that if the worker fails it will be get to know by the activity task and alliw another worker to grab it.

States that have long-running activities should provide a heartbeat timeout value to verify that the activity is still running successfully.

You can send heartbeat updates to the task.

If your activity has a heartbeat timeout value, the worker which implements it must send heartbeat updates to Step Functions. To send a heartbeat notification from a worker, use the SendTaskHeartbeat action.

If this task fails to complete within its time interval (if specified), or does not send heartbeat notifications in defined intervals, then the task is marked as failed. You can define the retry policy to try it again.

For more information refer SendTaskHeartbeat section in the AWS documentation.

Upvotes: 0

Related Questions