How to stop all running Step Functions of a specific state machine?

Question

I accidentally started very many step functions and now wish to terminate all of them.

Any smart ways to do this using the CLI or web console?

Answer 1

I tried Trisped script in linux and found out it needs to be converted in bash script so here it is in case you want to stop running executions in parallel from linux/unix.

#!/bin/bash

# Check if there are any running background jobs
if [[ $(jobs -r | wc -l) -gt 0 ]]; then
    echo "There are $(jobs -r | wc -l) jobs already running"
else
    echo "Remove all existing jobs."
jobs -r | xargs kill

StateMachineArn="<Step Function ARN>"
StateMachineRegion="<Step Function Region>"

# Define the function to stop the execution
ExecutionBlock() {
    StepFunctionExecutionArn=$1
    aws stepfunctions stop-execution --execution-arn "$StepFunctionExecutionArn" --region "$StateMachineRegion"
}

echo "Getting Step Function Execution ARNs for state machine with arn = '$StateMachineArn' in region = '$StateMachineRegion'."
StepFunctionExecutionArns=$(aws stepfunctions list-executions --state-machine-arn "$StateMachineArn" --status-filter RUNNING --query "executions[*].{executionArn:executionArn}" --output text --region "$StateMachineRegion")

MaxThreads=64
echo "Starting the jobs. Max $MaxThreads jobs running simultaneously."

for StepFunctionExecutionArn in $StepFunctionExecutionArns; do
    echo "Starting execution of job with input '$StepFunctionExecutionArn'."

    while [[ $(jobs -r | wc -l) -ge $MaxThreads ]]; do
        sleep 0.015
    done

    ExecutionBlock "$StepFunctionExecutionArn" &
    echo "Job with input '$StepFunctionExecutionArn' started."
done

echo "Waiting for jobs to finish."
wait

echo "Writing information from each job."
jobs -p | xargs -n 1 -I {} sh -c "echo 'Output from job {}: '; tail -n 1 /dev/stdin <<< \$(cat)"

echo "Cleaning up all jobs."
jobs -r | xargs kill
fi

Answer 2

I had 45,000 executions, so deleting using Pål Brattberg's answer was taking a long time, so I wrote a PowerShell script which will run the stop-execution command in parallel:

$ExecutionBlock = {
    Param([string] $StepFunctionExecutionArn)
    aws stepfunctions stop-execution --execution-arn $StepFunctionExecutionArn   
}

if ($(Get-Job -state running).count -gt 1) {
    Write-Host "There are $($(Get-Job -state running).count) jobs already running"
}
else {
    Write-Host 'Remove all existing jobs.'
    Get-Job | Remove-Job

    $StateMachineArn = "<Step Function ARN>"
    $StateMachineRegion = "<Step Function Region>"
    Write-Host "Getting Step Function Execution ARNs for state machine with arn = '$StateMachineArn' in region = '$StateMachineRegion'."
    [array]$StepFunctionExecutionArns = aws stepfunctions list-executions --state-machine-arn $StateMachineArn --status-filter RUNNING --query "executions[*].{executionArn:executionArn}" --output text --region $StateMachineRegion

    $MaxThreads = 64
    Write-Host "Starting the jobs. Max $MaxThreads jobs running simultaneously."
    foreach($StepFunctionExecutionArn in $StepFunctionExecutionArns){
        Write-Host "Starting execution of job with input '$StepFunctionExecutionArn'."

        While ($(Get-Job -state running).count -ge $MaxThreads) { Start-Sleep -Milliseconds 15 }
        Start-Job -Scriptblock $ExecutionBlock -ArgumentList $StepFunctionExecutionArn

        Write-Host "Job  with input '$StepFunctionExecutionArn' started."
    }


    Write-Host 'Waiting for jobs to finish.'
    Get-Job -state running | Wait-Job

    Write-Host 'Writing information from each job.'
    foreach($job in Get-Job) { Write-Host  Receive-Job -Id ($job.Id) }

    Write-Host "Cleaning up all jobs."
    Get-Job | Remove-Job
}

To use the script, replace <Step Function ARN> and <Step Function Region> with the correct values.

My results were 5-6 times faster than serial execution. Tweaking $MaxThreads may give better results.

Notes:

• You can check how many executions are still running in the web console by filtering on Running and checking the number next to Executions (may need to click the Load More button until you get all of them).
• You may want to change the logging. For example, add the number of executions in the $StepFunctionExecutionArns array.

Answer 3

for some reason after each iteration, it stuck in Mac,

adding >> out.t solves it

aws stepfunctions list-executions \
--state-machine-arn arn:aws:states:us-east-1:322348515048:stateMachine:workflow-dev-acknowledge-Awaiter \
--status-filter RUNNING \
--query "executions[*].{executionArn:executionArn}" \
--output text | \
xargs -I {} aws stepfunctions stop-execution \
--execution-arn {} >> out.t

Answer 4

For me xargs was giving issue because my execution-arn was quite big enough.

aws stepfunctions list-executions \
  --state-machine-arn <ARN> \
  --status-filter RUNNING \
  --query "executions[*].{executionArn:executionArn}" \
  --output text | \
    awk '{print}' |
    while read line;
    do aws stepfunctions stop-execution --execution-arn  $line
    done

This did the trick for me. Thanks to @Pål Brattberg

Answer 5

OK, let's do this using the CLI.

You can stop an execution using the following:

aws stepfunctions stop-execution \
  --execution-arn <STEP FUNCTION EXECUTION ARN>

But since I started way too many executions, it's helpful to be able to list all running executions of a state machine:

aws stepfunctions list-executions \
  --state-machine-arn <STEP FUNCTION ARN> \
  --status-filter RUNNING \
  --output text

Next, make sure to only list execution ARN's for these executions and list each execution ARN on a separate line:

aws stepfunctions list-executions \
  --state-machine-arn <STEP FUNCTION ARN> \
  --status-filter RUNNING \
  --query "executions[*].{executionArn:executionArn}" \
  --output text

Now, we put this together into one command using xargs:

aws stepfunctions list-executions \
  --state-machine-arn <STEP FUNCTION ARN> \
  --status-filter RUNNING \
  --query "executions[*].{executionArn:executionArn}" \
  --output text | \
xargs -I {} aws stepfunctions stop-execution \
  --execution-arn {} 

Now all running executions should be shut down. Make sure you do this with care so that you don't mess up production!

On that note, if you user aws-vault to minimize that very risk, the command above would look something like this:

aws-vault exec test-env -- aws stepfunctions list-executions \
  --state-machine-arn <STEP FUNCTION ARN> \
  --status-filter RUNNING \
  --query "executions[*].{executionArn:executionArn}" \
  --output text | \
xargs -I {} aws-vault exec test-env -- aws stepfunctions stop-execution \
  --execution-arn {}