CODEWITHSUNDEEP
amazon-web-serviceslambdaaws-lambdaaws-api-gatewayaws-glue
Sid Ali
Sid Ali

Reputation: 1857

Long time processing a lambda function

I have a lambda function which basically converts 'CSV files' into 'JSON files',

the problem is depending on the file the execution can take 5 sec or 400 sec maybe more,

Do you think it's, be an excellent solution to use lambda for this case, configuring the timeout for 10 min or something really high?

Upvotes: 0

Views: 581

Answers (2)

Sid Ali
Sid Ali

Reputation: 1857

After a meeting with an AWS technical advisor, i had this solutions :

  1. AWS batch
  2. AWS Glue

the AWS GLUE might be a better solution, it's an ETL and it was introduced by AWS for this kind of problem.

So the solution will be an AWS LAMBDA function which calls the AWS GLUE to transform a file from AWS S3, i'll call the LAMBDA through AWS API GATEWAY.

Upvotes: 0

Stephen
Stephen

Reputation: 1805

The maximum runtime for a Lambda function is 5 minutes (300s). So if you expect your runtimes might exceed this, Lambda is not a suitable technology to use. An AWS product like Batch or Fargate ECS might be more suitable.

Upvotes: 3

Related Questions