CODEWITHSUNDEEP
node.jsamazon-web-servicesdockeramazon-efsaws-copilot
HieroB
HieroB

Reputation: 4417

Can Docker containers running a node.js service on ECS in production share a node_modules volume mounted from EFS?

Is it good practice for node.js service containers running under AWS ECS to mount a shared node_modules volume persisted on EFS? If so, what's the best way to pre-populate the EFS before the app launches?

My front-end services run a node.js app, launched on AWS Fargate instances. This app uses many node_modules. Is it necessary for each instance to install the entire body of node_modules within its own container? Or can they all mount a shared EFS filesystem containing a single copy of the node_modules?

I've been migrating to AWS Copilot to orchestrate, but the docs are pretty fuzzy on how to pre-populate the EFS. At one point they say, "we recommend mounting a temporary container and using it to hydrate the EFS, but WARNING: we don't recommend this approach for production." (Storage: AWS Copilot Advanced Use Cases)

Upvotes: 1

Views: 721

Answers (2)

Austin Ely
Austin Ely

Reputation: 390

Thanks for the question! This is pointing out some gaps in our documentation that have been opened as we released new features. There is actually a manifest field, image.depends_on which mitigates the issue called out in the docs about prod usage.

To answer your question specifically about hydrating EFS volumes prior to service container start, you can use a sidecar and the image.depends_on field in your manifest.

For example:

image:
  build: ./Dockerfile
  depends_on:
    bootstrap: success

storage:
  volumes:
    common:
      path: /var/copilot/common
      read_only: true
      efs: true

sidecars:
  bootstrap:
    image: public.ecr.aws/my-image:latest
    essential: false #Allows the sidecar to run and terminate without the ECS task failing 
    mount_points:
      - source_volume: common
        path: /var/copilot/common
        read_only: false

On deployment, you'd build and push your sidecar image to ECR. It should include either your packaged data or a script to pull down the data you need, then move it over into the EFS volume at /var/copilot/common in the container filesystem.

Then, when you next run copilot svc deploy, the following things will happen:

  1. Copilot will create an EFS filesystem in your environment. Each service will get its own isolated access point in EFS, which means service containers all share data but can't see data added to EFS by other services.
  2. Your sidecar will run to completion. That means that all currently running services will see the changes to the EFS filesystem whenever a new copy of the task is deployed unless you specifically create task-specific subfolders in EFS in your startup script.
  3. Once the sidecar exits successfully, the new service container will come up on ECS and operate as normal. It will have access to the EFS volume which will contain the latest copy of your startup data.

Hope this helps.

Upvotes: 3

Mark B
Mark B

Reputation: 200998

Is it good practice for node.js service containers running under AWS ECS to mount a shared node_modules volume persisted on EFS?

Asking if it is "good" or not is a matter of opinion. It is generally a common practice in ECS however. You have to be very cognizant of the IOPS your application is going to generate against the EFS volume however. Once an EFS volume runs out of burst credits it can really slow down and impact the performance of your application.

I have not ever seen an EFS volume used to store node_modules before. In all honesty it seems like a bad idea to me. Dependencies like that should always be bundled in your Docker image. Otherwise it's going to be difficult when it comes time to upgrade those dependencies in your EFS volume, and may require down-time to upgrade.

If so, what's the best way to pre-populate the EFS before the app launches?

You would have to create the initial EFS volume and mount it somewhere like an EC2 instance, or another ECS container, and then run whatever commands necessary in that EC2/ECS instance to copy your files to the volume.

The quote in your question isn't present on the page you linked, so it's difficult to say exactly what other approach would the Copilot team would recommend.

Upvotes: 1

Related Questions