Reputation: 632
I have a sequence of data processing jobs (a pipeline) that I am executing using AWS Batch, and I would like some of these jobs to share the same EBS volumes. I dynamically create a new EBS volume specifically for each pipeline of Batch jobs to be executed. I can list, attach and mount EBS volumes using a script that runs at instance’s startup, written in the user data
field of the Launch Templates used by my Batch Compute Environments.
To be clear, in the user data
script of the Launch Template I can run commands such as:
aws ec2 describe-volumes
aws ec2 attach-volume
My problem is: I want to be able to choose, among a list of available EBS volumes, which one should be attached to the instance running a particular job. This will be known at the batch job submission time, but I can’t find a way to get access to this information at the ec2 startup time, i.e. within the user data
script.
Any ideas on how to accomplish that?
obs.: I would like to avoid using EFS, because of its costs.
Upvotes: 0
Views: 235