CODEWITHSUNDEEP
dockeroptimizationcontainerscloud
TBP
TBP

Reputation: 3

Best way to run dockers parallelly

I have a docker image that receives an argument and runs for roughly a minute.
From time to time I need to run it over a set of 10K-100K inputs.
I tried doing this using AWS Batch, but it ran very slowly, since at each given moment only few dockers ran.

Is there an easy alternative which allows configuring number of dockers to run simultaneously and thus controlling the over all run time?

Upvotes: 0

Views: 55

Answers (1)

lotjuh
lotjuh

Reputation: 191

As of December 2020, you can now run your docker containers on AWS Lambda: https://aws.amazon.com/blogs/aws/new-for-aws-lambda-container-image-support/

With this release, the maximum time a Lambda can run has been increased to 15 minutes (up from 5 minutes). Since you indicate your process only runs for roughly 1 minute, Lambda could be an option for you.

Upvotes: 2

Related Questions