Deploying multi-container docker application to AWS ECS Fargate using ECS-CLI

Question

I need to deploy this project on AWS ECS (Preferably Fargate or EC2 worst case). Looking at the documentation I tried to deploy with single container and it works but with multi containers, due to the restrictions of ecs-cli I cannot use the docker-compose.yml straight from project hence I upload the docker images to ECR and then create a new docker-compose with the digests for respective conatiners.

Here is a link to the original docker-compose.yml. Here is what my docker-compose looks like now after uploading images to ECR:

version: "3.0"
services:
    postgres:
        image: postgres:12  
        logging:
            driver: awslogs
            options: 
                awslogs-group: aws-ecs-docker-test
                awslogs-region: ap-south-1
                awslogs-stream-prefix: docker

    db:
        image: sha256:123123123123123213213213213
        logging:
            driver: awslogs
            options: 
                awslogs-group: aws-ecs-docker-test
                awslogs-region: ap-south-1
                awslogs-stream-prefix: docker   

    traefik:
        image: sha256:123123123123123213213213213
        logging:
            driver: awslogs
            options: 
                awslogs-group: aws-ecs-docker-test
                awslogs-region: ap-south-1
                awslogs-stream-prefix: docker

    queue:
        image: sha256:123123123123123213213213213
        logging:
            driver: awslogs
            options: 
                awslogs-group: aws-ecs-docker-test
                awslogs-region: ap-south-1
                awslogs-stream-prefix: docker
    
    flower:
        image: sha256:123123123123123213213213213
        env_file: 
            - .env
        logging:
            driver: awslogs
            options: 
                awslogs-group: aws-ecs-docker-test
                awslogs-region: ap-south-1
                awslogs-stream-prefix: docker

    backend:
        image: sha256:123123123123123213213213213
        env_file: 
            - .env
        environment:
            - SERVER_NAME=${DOMAIN?Variable not set}
            - SERVER_HOST=https://${DOMAIN?Variable not set}
            - SMTP_HOST=${SMTP_HOST}
        ports:
            - "80:80"
        logging:
            driver: awslogs
            options: 
                awslogs-group: aws-ecs-docker-test
                awslogs-region: ap-south-1
                awslogs-stream-prefix: docker

    celeryworker:
        image: sha256:123123123123123213213213213
        env_file:
            - .env
        environment:
            - SERVER_NAME=${DOMAIN?Variable not set}
            - SERVER_HOST=https://${DOMAIN?Variable not set}
            # Allow explicit env var override for tests
            - SMTP_HOST=${SMTP_HOST?Variable not set}
        logging:
            driver: awslogs
            options: 
                awslogs-group: aws-ecs-docker-test
                awslogs-region: ap-south-1
                awslogs-stream-prefix: docker
    

    frontend:
        image: sha256:123123123123123213213213213
        logging:
            driver: awslogs
            options: 
                awslogs-group: aws-ecs-docker-test
                awslogs-region: ap-south-1
                awslogs-stream-prefix: docker
    
volumes:
  app-db-data:

Here is the ecs-params.yml:

version: 1
task_definition:
  task_execution_role: ecsTaskExecutionRole
  ecs_network_mode: awsvpc
  task_size:
    mem_limit: 0.5GB
    cpu_limit: 256
run_params:
  network_configuration:
    awsvpc_configuration:
      subnets:
        - subnet-123123123
        - subnet-123123123
      security_groups:
        - sg-123123123
      assign_public_ip: ENABLED

The ecsTaskExecutionRole has all access to ECS, ECR & Cloudwatch logs. However whenever I deploy, ecs creates a few task definitions and then times out:

Deployment has not completed: Running count has not changed for 5.00 minutes

Even if I extend the timeout to 30 mins it doesn't change the output. The logs output nothing so I am quite clueless as to what could be the potential issue. I am new to Devops & Docker so I'm not sure what I am actually missing.

Answer 1

Alternatively to ecs-cli, I would suggest ECS Compose-X which will allow you to plug&play to your existing network (VPC) and takes care of all the rest (IAM, Security Groups etc). Additionally if you wanted to link your services to other AWS resources, you can use it to discover these resources (if already exist) or create new ones, and again, everything with regards to IAM and Security will be taken care of for you.

If you created, say, a kinesis stream, your container also would be given the ARN and name of the stream via env var automatically so you never need to name your resources, you'd always have a pointer to it.

Answer 2

I was able to fix this issue eventually. The issue was with the traefik image and the lack of Cloudwatch permissions given to the IAM role.