CODEWITHSUNDEEP
javaasynchronousamazon-s3aws-lambda
Awad
Awad

Reputation: 921

How to increase the timeout for AWS Lambda client

I am trying to scan a file using AWS Lambda, and I am getting timeout since the scan function is taking longer than expected.

I would like to increase the timeout for my client since this process is @Async and I can handle few more seconds.

This is my method:

   @Override
   @Async
   public void scanFile( String s3Bucket, String fileName, String path, String documentId, String createdBy ) throws IOException {

      FileScanInput input = new FileScanInput();
      input.setS3Bucket( s3Bucket );
      input.setS3Key( path );

      logger.debug( "Scan file: " + path + ", in S3 bucket:  " + s3Bucket );
      if ( fileScanService == null ) {
         fileScanService = buildFileScanService();
      }

      FileScanOutput fileScanOutput = fileScanService.scanFile( input );
//      TODO: if the scan process failed, ask what should be the next step.
//      for now, the file stays in S3.
      Assert.notNull( fileScanOutput );
      String status = fileScanOutput.getStatus();

      // in case the document owner was not found an infected file was file. Email approved emails
      if ( status.equalsIgnoreCase( VIRUS_SCAN_INFECTED ) ) {
         // delete file on S3
         this.deleteFile( s3Bucket, path );
         String toEmail = personInfoMapper.findStudentEmail( createdBy );
         try {
            sendVirusDetectedEmail( fileName, toEmail );
         }
         catch ( Exception e ) {
            logger.error( e.getMessage() );
         }

         //         we clean up the metadata table in case there was a virus since this is running async.
         metadataDao.cleanUpMetadata( documentId );

         logger.error( "The file is infected." );
         throw new VirusFoundException( "File is infected." );
      }
   }


   public final FileScanService buildFileScanService() {
      return LambdaInvokerFactory.builder().lambdaClient( AWSLambdaClientBuilder.defaultClient() ).build( FileScanService.class );
   }

And this is the resource configs for my Lambda function. enter image description here

Update I also noticed that my Lambda function actually does its job, which means the issue is basically in this line FileScanOutput fileScanOutput = fileScanService.scanFile( input );

fileScanOutput doesn't get initialized instead I get timeout issue.

My other classes look like:

public interface FileScanService {

   @LambdaFunction( functionName = "s3-antivirus-api-scan" )
   public FileScanOutput scanFile( FileScanInput fileScanInput );
}

public class FileScanInput {

   private String s3Bucket;
   private String s3Key;

   public String getS3Bucket() {
      return s3Bucket;
   }

   public void setS3Bucket( String value ) {
      s3Bucket = value;
   }

   public String getS3Key() {
      return s3Key;
   }

   public void setS3Key( String value ) {
      s3Key = value;
   }
}



public class FileScanOutput {

   private String status;

   public FileScanOutput() {
   }

   public FileScanOutput( String status ) {
      this.status = status;
   }

   public String getStatus() {
      return status;
   }

   public void setStatus( String value ) {
      status = value;
   }
}

Upvotes: 10

Views: 37606

Answers (6)

Drew H
Drew H

Reputation: 1292

I just had this issue and solved it.

Turns out that if the lambda is behind an API Gateway, the AWS has a hard 29-second timeout for REST requests. The Gateway can terminate the connection with a 504: "Endpoint request timed out" response. To see limits on AWS Gateways see https://docs.aws.amazon.com/apigateway/latest/developerguide/limits.html#api-gateway-execution-service-limits-table

A workaround to this issue is to have the Gateway execute the lambda asynchronously. To have the Gateway execute the lambda asynchronously add the following header to the REST request.

"X-Amz-Invocation-Type": "Event"

This worked for me. See also https://docs.aws.amazon.com/apigateway/latest/developerguide/set-up-lambda-integration-async.html

Upvotes: 0

F.H.
F.H.

Reputation: 1664

If the timeout occured after 120 seconds it is probably the sdk client which timed out.

This did the trick for me (nodejs example):

import aws from 'aws-sdk';

const lambdaHandler = new aws.Lambda({
    ...
    httpOptions: {
        timeout: 15 * 60 * 1000 // 15 minutes
    }, 
});

You can see the available constructor parameters here: https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/Lambda.html#updateFunctionConfiguration-property

Upvotes: 1

jalogar
jalogar

Reputation: 1684

When using the new sdk, for increasing the timeout in the client, you can do something like this:

SdkHttpClient httpClient = ApacheHttpClient.builder()
            .socketTimeout(Duration.ofSeconds(50))
            .build();

LambdaClient lambdaClient = LambdaClient.builder()
            .httpClient(httpClient)
            .build();

Notice there is also a ClientOverrideConfiguration that could be passed to create the lambdaClient, which has methods apiCallTimeout and apiCallAttemptTimeout, but in my case it only started working when I set the timeout in the httpClient.

Upvotes: 2

gil.fernandes
gil.fernandes

Reputation: 14591

In order to increase the timeout in 2021 you will need to:

  • edit your function
  • click on the configuration tab enter image description here
  • Click on the Edit button
  • Change the timeout property in the Basic Settings category: enter image description here
  • Press Save down below

Upvotes: 7

hoangdv
hoangdv

Reputation: 16127

AWS Lambda function has maximum execution duration per request is 900 seconds (15 minutes), the default value is 30 seconds.

You can increase this value in function settings:

Open the AWS Lambda console at https://console.aws.amazon.com/lambda/

Choose your function.

Change the Timeout setting.

Timeout setting

Upvotes: 4

pbeardshear
pbeardshear

Reputation: 1000

When you say your client is timing out, do you mean your Lambda SDK client? If so, you may need to pass a longer socket timeout when creating your client:

AWSLambdaClientBuilder.standard()
  .withClientConfiguration(new ClientConfiguration()
    .withSocketTimeout(SOCKET_TIMEOUT_IN_MS))
  .build();

The default socket timeout is 50 seconds: https://github.com/aws/aws-sdk-java/blob/master/aws-java-sdk-core/src/main/java/com/amazonaws/ClientConfiguration.java#L43

Your Lambda function itself will continue running regardless of whether the socket is closed on the client side, which is likely why you see the function completing the job.

Upvotes: 9

Related Questions