CODEWITHSUNDEEP
amazon-web-servicesboto3aws-step-functions
Noam
Noam

Reputation: 191

AWS StepFunctions - SendTaskSuccess/SendTaskFailure operation - Task time out

I'm trying to execute a step function that has a task that uses an activity resource.

and after running my code i would like to return a SendTaskSuccess or SendTaskFailure

back to the step function.

I'm getting the following error sporadically (after less then 10 seconds):

An error occurred (TaskTimedOut) when calling the SendTaskFailure operation: Task Timed Out: 'arn:aws:states:us-east-1:<....>'

where do i disable the TaskTimeOut? or how do i increase it?

more info

This is the step function code

{
  "Comment": "orchestrator-state-machine",
  "StartAt": "get_data_from_lambda",
  "States": {

    "get_data_from_lambda": {
      "Type": "Pass",
      "Next": "start_task"
    },

    "start_task": {
      "Type": "Task",
      "Resource": "arn:aws:states:us-east-1:<...>",
      "End": true
    }

  }
}

Code for sending success

self._client = boto3.client('stepfunctions', self._region, config=Config(connect_timeout=65,read_timeout=70,region_name=self._region))

...
def task_success(self, token, result):
    self.logger.info(str(token))
    try:
        self._client.send_task_success(taskToken=token, output=result)
        self.logger.info("sending success back to step function")
    except Exception as e:
        self.logger.error("did not send success to step function: {0}".format(str(e)))

    return True

request details

{
  'Error': {
    'Message': "Task Timed Out: 'arn:aws:states:us-east-1:<....>t'",
    'Code': 'TaskTimedOut'
  },
  'ResponseMetadata': {
    'RequestId': 'b******d-755a-****-9cef-ad*******a4c',
    'HTTPStatusCode': 400,
    'HTTPHeaders': {
      'x-amzn-requestid': 'b******d-755a-****-9cef-ad*******a4c',
      'content-type': 'application/x-amz-json-1.0',
      'content-length': '154'
    },
    'RetryAttempts': 0
  }
}

Upvotes: 7

Views: 8571

Answers (3)

lynkfox
lynkfox

Reputation: 2400

It may not apply here, but if your Step Function completes in under 30 seconds, you can use an Express Step Function (and boto3.resource('stepfunction).start_sync_execution) in order to wait for the response.

But I see from other responses you have that you may have very long response times.

The other option is to flip it, and use a script in your EC2 instance that you mentioned that polls the step function regularly (using the describe_execution boto3 method) and returns once it has a successful response.

Upvotes: 0

Shital
Shital

Reputation: 77

You can add timeout to activity step by adding Timeout=300 where 300 would be time in seconds. "start_task": { "Type": "Task", "Resource": "arn:aws:states:us-east-1:<...>", "Timeout":300, "End": true } Also you need make your the resouce arn is activity arn and not lambda function arn

Upvotes: 0

Noam
Noam

Reputation: 191

I checked around and managed to reproduce the issue quite easily using AWS command line interface

Seems like it's an issue with AWS and it's now resolved.

Upvotes: 0

Related Questions