CODEWITHSUNDEEP
amazon-web-servicesarchitectureaws-lambda
ajshort
ajshort

Reputation: 3754

Long-running AWS Lambda tasks with progress and cancellation

I have an application where I'm looking to offload the compute load to AWS, and am after some guidance on architecture. The user will initiate a main task, which contains ~100 computationally-heavy sub-tasks which can be run in parallel.

I am thinking an appropriate solution is for the desktop app to hit an API gateway endpoint to create a new task, which would then invoke many Lambdas, one for each sub-task. I would like each sub-task to have individual progress reporting, as well as the ability for the user to cancel the overall task. The user could also use the API to query the created task / hit another endpoint to cancel it.

What's an appropriate architecture / service(s) to invoke and manage these Lambda sub-tasks, access intermediate progress information from each Lambda, the final result, and allow the user to request cancelation?

Upvotes: 1

Views: 2533

Answers (1)

Icehorn
Icehorn

Reputation: 1337

You may be interested in AWS Step Functions (https://aws.amazon.com/step-functions/) for the querying and orchestration of the overall progress, and possibly use DynamoDB (https://aws.amazon.com/dynamodb/) or some other data store to allow for monitoring the progress within individual sub tasks.

Upvotes: 1

Related Questions