How to create an ECS task in CloudFormation before the CodePipeline is created

Question

I'm trying to define my ECS stack in Cloudformation, including the CI/CD pipeline and ECR repository. However you run into a bit of a conundrum in that:

1. To create an ECS task definition (AWS::ECS::TaskDefinition) you have to first create a populated ECR repository (AWS::ECR::Repository) so that you can specify the Image property.
2. To populate this repository you have to first create the CodePipeline (AWS::CodePipeline::Pipeline) which will run automatically on creation.
3. To create the pipeline you have to first create the ECS task definition / cluster as the pipeline needs to deploy onto it (back to step 1).

The solutions to this I can see are:

• Don't create the ECR repository in Cloudformation & pass it as a parameter to the stacks.
• Define a dummy image in the task definition to deploy the first time and then create the pipeline which will create the real ECR repository and deploy the real image.
• Create the CodeBuild project and ECR repository in a separate stack, trigger the CodeBuild project with a lambda function (I don't think it runs automatically on creation like the pipeline does), create the ECS cluster and then create the pipeline. This seems more complicated than it should be.

Are there any better ways of approaching this problem?

Answer 1

We decided to solve this by making a proper ECS Minimal Container, which can be deployed immediately and will respond to health checks immediately. We wrapped it up into a GitHub repository and made a wizard script for setting this up:

https://github.com/perholmes/ECSMinimalContainer

This script simply automates creating a bare-bones Apache container that responds to health checks. It's uploaded to an Elastic Container Registry ahead of time, and can then be referenced in TaskDefinitions immediately. The containers will boot, allowing the CloudFormation to finish deploying. Both the cluster and any target groups will respond to health checks.

We needed this for a big combo CloudFormation that deployed clusters, build pipelines and services in one go, which was impossible without some kind of dummy container that can result in a stable cluster immediately.

Answer 2

Another way to create this by using a single stack is to trigger the Fargate deployment after pushing the image by making an initial commit to the CodeCommit repository and setting the DesiredCount property of the ECS service to zero:

Repo:
  Type: AWS::CodeCommit::Repository
  Properties:
    Code:
      BranchName: main
      S3:
        Bucket: some-bucket
        Key: code.zip
    RepositoryName: !Select [4, !Split ['-', !Select [2, !Split ['/', !Ref AWS::StackId]]]]
    RepositoryDescription: Repository
    Triggers:
      - Name: Trigger
        CustomData: The Code Repository
        DestinationArn: !Ref Topic
        Branches:
          - main
        Events: [all]

Service:
  Type: AWS::ECS::Service
  Properties:
    Cluster: !Ref Cluster
    DesiredCount: 0
    LaunchType: FARGATE
    NetworkConfiguration:
      AwsvpcConfiguration:
        SecurityGroups:
          - !Ref SecG
        Subnets: !Ref Subs
    ServiceName: !Select [4, !Split ['-', !Select [2, !Split ['/', !Ref AWS::StackId]]]]
    TaskDefinition: !Ref TaskDefinition

  Build:
    Type: AWS::CodeBuild::Project
    Properties:
      Artifacts:
        Type: CODEPIPELINE
      Source:
        Type: CODEPIPELINE
        BuildSpec: !Sub |
          version: 0.2
          phases:
            pre_build:
              commands:
                - echo "[`date`] PRE_BUILD"
                - echo "Logging in to Amazon ECR..."
                - aws ecr get-login-password --region $REGION | docker login --username AWS --password-stdin $ACCOUNT.dkr.ecr.$REGION.amazonaws.com
                - IMAGE_URI="$ACCOUNT.dkr.ecr.$REGION.amazonaws.com/$REPO:$TAG"
            build:
              commands:
                - echo "[`date`] BUILD"
                - echo "Building Docker Image..."    
                - docker build -t $REPO:$TAG .
                - docker tag $REPO:$TAG $IMAGE_URI
            post_build:
              commands:
                - echo "[`date`] POST_BUILD"
                - echo "Pushing Docker Image..."
                - docker push $IMAGE_URI
                - echo Writing image definitions file...
                - printf '[{"name":"svc","imageUri":"%s"}]' $IMAGE_URI > $FILE
          artifacts:
              files: $FILE
      Environment:
        ComputeType: BUILD_GENERAL1_SMALL
        Image: aws/codebuild/standard:6.0
        Type: LINUX_CONTAINER
        EnvironmentVariables:
          - Name: REGION
            Type: PLAINTEXT
            Value: !Ref AWS::Region
          - Name: ACCOUNT
            Type: PLAINTEXT
            Value: !Ref AWS::AccountId
          - Name: TAG
            Type: PLAINTEXT
            Value: latest
          - Name: REPO
            Type: PLAINTEXT
            Value: !Ref Registry
          - Name: FILE
            Type: PLAINTEXT
            Value: !Ref ImagesFile
        PrivilegedMode: true
      Name: !Ref AWS::StackName
      ServiceRole: !GetAtt CodeBuildServiceRole.Arn

  Pipeline:
    Type: AWS::CodePipeline::Pipeline
    Properties:
      RoleArn: !GetAtt CodePipelineServiceRole.Arn
      ArtifactStore:
        Type: S3
        Location: !Ref ArtifactBucket
      Stages:
        - Name: Source
          Actions:
            - Name: Site
              ActionTypeId:
                Category: Source
                Owner: AWS
                Version: '1'
                Provider: CodeCommit
              Configuration:
                RepositoryName: !GetAtt Repo.Name
                BranchName: main
                PollForSourceChanges: 'false'
              InputArtifacts: []
              OutputArtifacts:
                - Name: SourceArtifact
              RunOrder: 1
        - Name: Build
          Actions:
            - Name: Docker
              ActionTypeId:
                Category: Build
                Owner: AWS
                Version: '1'
                Provider: CodeBuild
              Configuration:
                ProjectName: !Ref Build
              InputArtifacts:
                - Name: SourceArtifact
              OutputArtifacts:
                - Name: BuildArtifact
              RunOrder: 1
        - Name: Deploy
          Actions:
            - Name: Fargate
              ActionTypeId:
                Category: Deploy
                Owner: AWS
                Version: '1'
                Provider: ECS
              Configuration:
                ClusterName: !Ref Cluster
                FileName: !Ref ImagesFile
                ServiceName: !GetAtt Service.Name
              InputArtifacts:
                - Name: BuildArtifact
              RunOrder: 1

Note that the some-bucket S3 bucket needs to contain the zipped .Dockerfile and any source code without any .git directory included.

If you use another service for your repo, like GitHub for instance, or your already have a repo, simple remove the section and configure the pipeline as required.

The entire CloudFormation stack is listed below for reference:

AWSTemplateFormatVersion: '2010-09-09'
Description: CloudFormation Stack to Trigger CodeBuild via CodePipeline

Parameters:
  SecG:
    Description: Single security group
    Type: AWS::EC2::SecurityGroup::Id
  Subs:
    Description: Comma separated subnet IDs
    Type: List<AWS::EC2::Subnet::Id>
  ImagesFile:
    Type: String
    Default: images.json

Resources:
  ArtifactBucket:
    Type: AWS::S3::Bucket
    DeletionPolicy: Retain
    Properties:
      PublicAccessBlockConfiguration:
        BlockPublicAcls: true
        BlockPublicPolicy: true
        IgnorePublicAcls: true
        RestrictPublicBuckets: true
      Tags:
        - Key: UseWithCodeDeploy
          Value: true

  CodeBuildServiceRole:
    Type: AWS::IAM::Role
    Properties:
      Path: /
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service: codebuild.amazonaws.com
            Action: sts:AssumeRole
      Policies:
        - PolicyName: !Sub 'ssm-${AWS::Region}-${AWS::StackName}'
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
                -
                  Effect: Allow
                  Action:
                      - ssm:GetParameters
                      - secretsmanager:GetSecretValue
                  Resource: '*'
        - PolicyName: !Sub 'logs-${AWS::Region}-${AWS::StackName}'
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
                -
                  Effect: Allow
                  Action:
                      - logs:CreateLogGroup
                      - logs:CreateLogStream
                      - logs:PutLogEvents
                  Resource: '*'
        - PolicyName: !Sub 'ecr-${AWS::Region}-${AWS::StackName}'
          PolicyDocument:
            Version: "2012-10-17"
            Statement:
                -
                  Effect: Allow
                  Action:
                      - ecr:BatchCheckLayerAvailability
                      - ecr:CompleteLayerUpload
                      - ecr:GetAuthorizationToken
                      - ecr:InitiateLayerUpload
                      - ecr:PutImage
                      - ecr:UploadLayerPart
                      - lightsail:*
                  Resource: '*'
        - PolicyName: !Sub bkt-${ArtifactBucket}-${AWS::Region}
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
                -
                  Effect: Allow
                  Action:
                      - s3:ListBucket
                      - s3:GetBucketLocation
                      - s3:ListBucketVersions
                      - s3:GetBucketVersioning
                  Resource: 
                      - !Sub arn:aws:s3:::${ArtifactBucket}
                      - arn:aws:s3:::some-bucket
        - PolicyName: !Sub obj-${ArtifactBucket}-${AWS::Region}
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
                -
                  Effect: Allow
                  Action:
                      - s3:GetObject
                      - s3:PutObject
                      - s3:GetObjectAcl
                      - s3:PutObjectAcl
                      - s3:GetObjectTagging
                      - s3:PutObjectTagging
                      - s3:GetObjectVersion
                      - s3:GetObjectVersionAcl
                      - s3:PutObjectVersionAcl
                  Resource: 
                      - !Sub arn:aws:s3:::${ArtifactBucket}/*
                      - arn:aws:s3:::some-bucket/*
  CodeDeployServiceRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Statement:
          - Sid: '1'
            Effect: Allow
            Principal:
              Service:
                - codedeploy.us-east-1.amazonaws.com
                - codedeploy.eu-west-1.amazonaws.com
            Action: sts:AssumeRole
      Path: /
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/AWSCodeDeployRoleForECS
        - arn:aws:iam::aws:policy/service-role/AWSCodeDeployRole
        - arn:aws:iam::aws:policy/service-role/AWSCodeDeployRoleForLambda
  CodeDeployRolePolicies:
    Type: AWS::IAM::Policy
    Properties:
      PolicyName: !Sub 'CDPolicy-${AWS::Region}-${AWS::StackName}'
      PolicyDocument:
        Statement:
          - Effect: Allow
            Resource:
              - '*'
            Action:
              - ec2:Describe*
          - Effect: Allow
            Resource:
              - '*'
            Action:
              - autoscaling:CompleteLifecycleAction
              - autoscaling:DeleteLifecycleHook
              - autoscaling:DescribeLifecycleHooks
              - autoscaling:DescribeAutoScalingGroups
              - autoscaling:PutLifecycleHook
              - autoscaling:RecordLifecycleActionHeartbeat
      Roles:
        - !Ref CodeDeployServiceRole
  CodePipelineServiceRole:
    Type: AWS::IAM::Role
    Properties:
      Path: /
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              Service: codepipeline.amazonaws.com
            Action: sts:AssumeRole
      Policies:
        - PolicyName: !Sub 'root-${AWS::Region}-${AWS::StackName}'
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              - Resource:
                  - !Sub 'arn:aws:s3:::${ArtifactBucket}/*'
                  - !Sub 'arn:aws:s3:::${ArtifactBucket}'
                Effect: Allow
                Action:
                  - s3:PutObject
                  - s3:GetObject
                  - s3:GetObjectVersion
                  - s3:GetBucketAcl
                  - s3:GetBucketLocation
              - Resource: "*"
                Effect: Allow
                Action:
                  - ecs:*
              - Resource: "*"
                Effect: Allow
                Action:
                  - iam:PassRole
                Condition:
                  StringLike:
                    iam:PassedToService:
                      - ecs-tasks.amazonaws.com
              - Resource: !GetAtt Build.Arn
                Effect: Allow
                Action:
                  - codebuild:BatchGetBuilds
                  - codebuild:StartBuild
                  - codebuild:BatchGetBuildBatches
                  - codebuild:StartBuildBatch
              - Resource: !GetAtt Repo.Arn
                Effect: Allow
                Action:
                  - codecommit:CancelUploadArchive
                  - codecommit:GetBranch
                  - codecommit:GetCommit
                  - codecommit:GetRepository
                  - codecommit:GetUploadArchiveStatus
                  - codecommit:UploadArchive

  AmazonCloudWatchEventRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: '2012-10-17'
        Statement:
          -
            Effect: Allow
            Principal:
              Service:
                - events.amazonaws.com
            Action: sts:AssumeRole
      Path: /
      Policies:
        -
          PolicyName: cwe-pipeline-execution
          PolicyDocument:
            Version: '2012-10-17'
            Statement:
              -
                Effect: Allow
                Action: codepipeline:StartPipelineExecution
                Resource: !Sub arn:aws:codepipeline:${AWS::Region}:${AWS::AccountId}:${Pipeline}
  AmazonCloudWatchEventRule:
    Type: AWS::Events::Rule
    Properties:
      EventPattern:
        source:
          - aws.codecommit
        detail-type:
          - CodeCommit Repository State Change
        resources:
          - !GetAtt Repo.Arn
        detail:
          event:
            - referenceCreated
            - referenceUpdated
          referenceType:
            - branch
          referenceName:
            - main
      Targets:
        -
          Arn: !Sub arn:aws:codepipeline:${AWS::Region}:${AWS::AccountId}:${Pipeline}
          RoleArn: !GetAtt AmazonCloudWatchEventRole.Arn
          Id: codepipeline-Pipeline

  Topic:
    Type: AWS::SNS::Topic
    Properties:
      Subscription:
        - Endpoint: user@example.com
          Protocol: email
  TopicPolicy:
    Type: AWS::SNS::TopicPolicy
    Properties: 
      PolicyDocument:
        Version: '2012-10-17'
        Statement:
          - 
            Sid: AllowPublish
            Effect: Allow
            Principal:
              Service:
                - 'codestar-notifications.amazonaws.com'
            Action:
              - 'SNS:Publish'
            Resource:
              - !Ref Topic
      Topics: 
        - !Ref Topic
  Repo:
    Type: AWS::CodeCommit::Repository
    Properties:
      Code:
        BranchName: main
        S3:
          Bucket: some-bucket
          Key: code.zip
      RepositoryName: !Select [4, !Split ['-', !Select [2, !Split ['/', !Ref AWS::StackId]]]]
      RepositoryDescription: Repository
      Triggers:
        - Name: Trigger
          CustomData: The Code Repository
          DestinationArn: !Ref Topic
          Branches:
            - main
          Events: [all]
  RepoUser:
    Type: AWS::IAM::User
    Properties:
      Path: '/'
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/AWSCodeCommitPowerUser
  RepoUserKey:
    Type: AWS::IAM::AccessKey
    Properties:
      UserName:
        !Ref RepoUser

  Registry: 
    Type: AWS::ECR::Repository
    Properties: 
      RepositoryName: !Select [4, !Split ['-', !Select [2, !Split ['/', !Ref AWS::StackId]]]]
      RepositoryPolicyText: 
        Version: '2012-10-17'
        Statement:
          - Sid: AllowPushPull
            Effect: Allow
            Principal:
              AWS:  
                - !GetAtt CodeDeployServiceRole.Arn
            Action:
              - ecr:GetDownloadUrlForLayer
              - ecr:BatchGetImage
              - ecr:BatchCheckLayerAvailability
              - ecr:PutImage
              - ecr:InitiateLayerUpload
              - ecr:UploadLayerPart
              - ecr:CompleteLayerUpload
  Build:
    Type: AWS::CodeBuild::Project
    Properties:
      Artifacts:
        Type: CODEPIPELINE
      Source:
        Type: CODEPIPELINE
        BuildSpec: !Sub |
          version: 0.2
          phases:
            pre_build:
              commands:
                - echo "[`date`] PRE_BUILD"
                - echo "Logging in to Amazon ECR..."
                - aws ecr get-login-password --region $REGION | docker login --username AWS --password-stdin $ACCOUNT.dkr.ecr.$REGION.amazonaws.com
                - IMAGE_URI="$ACCOUNT.dkr.ecr.$REGION.amazonaws.com/$REPO:$TAG"
            build:
              commands:
                - echo "[`date`] BUILD"
                - echo "Building Docker Image..."    
                - docker build -t $REPO:$TAG .
                - docker tag $REPO:$TAG $IMAGE_URI
            post_build:
              commands:
                - echo "[`date`] POST_BUILD"
                - echo "Pushing Docker Image..."
                - docker push $IMAGE_URI
                - echo Writing image definitions file...
                - printf '[{"name":"svc","imageUri":"%s"}]' $IMAGE_URI > $FILE
          artifacts:
              files: $FILE
      Environment:
        ComputeType: BUILD_GENERAL1_SMALL
        Image: aws/codebuild/standard:6.0
        Type: LINUX_CONTAINER
        EnvironmentVariables:
          - Name: REGION
            Type: PLAINTEXT
            Value: !Ref AWS::Region
          - Name: ACCOUNT
            Type: PLAINTEXT
            Value: !Ref AWS::AccountId
          - Name: TAG
            Type: PLAINTEXT
            Value: latest
          - Name: REPO
            Type: PLAINTEXT
            Value: !Ref Registry
          - Name: FILE
            Type: PLAINTEXT
            Value: !Ref ImagesFile
        PrivilegedMode: true
      Name: !Ref AWS::StackName
      ServiceRole: !GetAtt CodeBuildServiceRole.Arn
  Pipeline:
    Type: AWS::CodePipeline::Pipeline
    Properties:
      RoleArn: !GetAtt CodePipelineServiceRole.Arn
      ArtifactStore:
        Type: S3
        Location: !Ref ArtifactBucket
      Stages:
        - Name: Source
          Actions:
            - Name: Site
              ActionTypeId:
                Category: Source
                Owner: AWS
                Version: '1'
                Provider: CodeCommit
              Configuration:
                RepositoryName: !GetAtt Repo.Name
                BranchName: main
                PollForSourceChanges: 'false'
              InputArtifacts: []
              OutputArtifacts:
                - Name: SourceArtifact
              RunOrder: 1
        - Name: Build
          Actions:
            - Name: Docker
              ActionTypeId:
                Category: Build
                Owner: AWS
                Version: '1'
                Provider: CodeBuild
              Configuration:
                ProjectName: !Ref Build
              InputArtifacts:
                - Name: SourceArtifact
              OutputArtifacts:
                - Name: BuildArtifact
              RunOrder: 1
        - Name: Deploy
          Actions:
            - Name: Fargate
              ActionTypeId:
                Category: Deploy
                Owner: AWS
                Version: '1'
                Provider: ECS
              Configuration:
                ClusterName: !Ref Cluster
                FileName: !Ref ImagesFile
                ServiceName: !GetAtt Service.Name
              InputArtifacts:
                - Name: BuildArtifact
              RunOrder: 1

  Cluster:
    Type: AWS::ECS::Cluster
    Properties:
      ClusterName: !Select [4, !Split ['-', !Select [2, !Split ['/', !Ref AWS::StackId]]]]
  FargateTaskExecutionRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - ecs-tasks.amazonaws.com
            Action:
              - sts:AssumeRole
      ManagedPolicyArns:
        - arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy
  TaskRole:
    Type: AWS::IAM::Role
    Properties:
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal:
              Service:
                - ecs-tasks.amazonaws.com
            Action:
              - sts:AssumeRole
  TaskDefinition:
    Type: AWS::ECS::TaskDefinition
    Properties:
      ContainerDefinitions:
        - 
          Name: svc
          Image: !Sub ${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/${Registry}:latest
          PortMappings: 
            - ContainerPort: 8080
      Cpu: 256
      ExecutionRoleArn: !Ref FargateTaskExecutionRole
      Memory: 512
      NetworkMode: awsvpc
      RequiresCompatibilities:
        - FARGATE
      RuntimePlatform:
        CpuArchitecture: ARM64
        OperatingSystemFamily: LINUX
      TaskRoleArn: !Ref TaskRole  
  Service:
    Type: AWS::ECS::Service
    Properties:
      Cluster: !Ref Cluster
      DesiredCount: 0
      LaunchType: FARGATE
      NetworkConfiguration:
        AwsvpcConfiguration:
          SecurityGroups:
            - !Ref SecG
          Subnets: !Ref Subs
      ServiceName: !Select [4, !Split ['-', !Select [2, !Split ['/', !Ref AWS::StackId]]]]
      TaskDefinition: !Ref TaskDefinition

Outputs:
  ArtifactBucketName:
    Description: ArtifactBucket S3 Bucket Name
    Value: !Ref ArtifactBucket
  ArtifactBucketSecureUrl:
    Description: ArtifactBucket S3 Bucket Domain Name
    Value: !Sub 'https://${ArtifactBucket.DomainName}'

  ClusterName:
    Value: !Ref Cluster
  ServiceName:
    Value: !GetAtt Service.Name

  RepoUserAccessKey:
    Description: S3 User Access Key
    Value: !Ref RepoUserKey
  RepoUserSecretKey:
    Description: S3 User Secret Key
    Value: !GetAtt RepoUserKey.SecretAccessKey

  BuildArn:
    Description: CodeBuild URL
    Value: !GetAtt Build.Arn
  RepoArn:
    Description: CodeCommit Repository ARN
    Value: !GetAtt Repo.Arn
  RepoName:
    Description: CodeCommit Repository NAme
    Value: !GetAtt Repo.Name
  RepoCloneUrlHttp:
    Description: CodeCommit HTTP Clone URL
    Value: !GetAtt Repo.CloneUrlHttp
  RepoCloneUrlSsh:
    Description: CodeCommit SSH Clone URL
    Value: !GetAtt Repo.CloneUrlSsh
  PipelineUrl:
    Description: CodePipeline URL
    Value: !Sub https://console.aws.amazon.com/codepipeline/home?region=${AWS::Region}#/view/${Pipeline}
  RegistryUri:
    Description: ECR Repository URI
    Value: !GetAtt Registry.RepositoryUri
  TopicArn:
    Description: CodeCommit Notification SNS Topic ARN
    Value: !Ref Topic

Hope this helps!

Answer 3

The way I do it is to ECR repository first, but still using CloudFormation. So I have two templates. One for ECR repo. And the second one for the rest. The ECR repo is passed as a parameter to the second template. But you can also export its Uri to be ImportValue in the second step. The Uri is created as follows:

  Uri:
    Value: !Sub "${AWS::AccountId}.dkr.ecr.${AWS::Region}.amazonaws.com/${MyECR}"

You will also need some initial image in the repo for the task definition. This you can automate by having separated CodeBuild project (no need for CodePipeline) for this initial build.