CODEWITHSUNDEEP
amazon-web-servicesnetwork-programmingaws-cloudformationamazon-ecs
jfefes
jfefes

Reputation: 13

Reusing one ALB in Cloudformation?

I'm in the process of migration multiple microservices into ECS with Fargate. These services are set up so that traffic is routed to them based on path pattern. To accomplish this, I think I can have one single ALB, with one HTTPS Listener, and multiple Listener Rules (one per service). Each microservice is in it's own repo (this is a work project), and our desired state is to have the repo-specific Cloudformation data stay it it's own repo.

My approach was to do the following:

Infrastructure repo: ALB, HTTPS Listener, Listener Roles

each microservice: Service, Task, TargetGroup, Logs, Task Scaling, etc.

I'm hitting a wall here, in that without making a listener for each target group, my target groups don't seem to be registering my ECS tasks, nor does my ALB seem to be registering with the ECS service. I know little about Custom Resources, is that something I would have to do in order to accomplish this? It seems like I cannot get this working without an additional piece.

For a visual idea of what i'm hoping to accomplish: https://www.lucidchart.com/publicSegments/view/a914fb18-fc46-4f9b-87d9-6d270afe9933/image.png

Upvotes: 1

Views: 1145

Answers (2)

lexicore
lexicore

Reputation: 43709

We do exactly as you have described without any custom resources.

Our infra is organized in 2+n templates roughly as follows:

  • LoadBalancer.template creates

    • LoadBalancer
    • LoadBalancerHttpListener
    • LoadBalancerHttpsListener
    • LoadBalancerSecurityGroup

  • ServicesCluster.template creates

    • ServiceCluster
    • ServiceClusterAutoScalingGroup
    • ServiceClusterCPUReservationHighAlert/ServiceClusterCPUReservationLowAlert
    • ServiceClusterCPUReservationScaleDownPolicy/ServiceClusterCPUReservationScaleUpPolicy
    • ServiceClusterInstanceProfile
    • ServiceClusterLaunchConfiguration
    • ServiceClusterLogGroup
    • ServiceClusterSecurityGroup
    • ServiceClusterSecurityGroupIngressForItself
    • ServiceDiscoveryNamespace

  • Per-microserviceSomeMicroservice.template creates:

    • SomeMicroserviceHttpListenerRule/SomeMicroserviceHttpsListenerRule - connected to ALB listener with path-pattern, forwarding to SomeMicroserviceTargetGroup
    • SomeMicroserviceTargetGroup
    • SomeMicroservice (ECS Service)
    • SomeMicroserviceTask

Here's a piece of template for your reference. I have tried to pseudonymize it, hope it's still valid:

Resources:
  SomeMicroserviceTask:
    Type: 'AWS::ECS::TaskDefinition'
    Properties:
      Family: !Sub 'stk-${EnvType}-${EnvId}-${Component}-task'
      ExecutionRoleArn: !ImportValue
        'Fn::Sub': 'stk-${EnvType}-${EnvId}-roles-ClusterInstanceRole'
      Memory: 256
      ContainerDefinitions:
        - Name: !Sub 'stk-${EnvType}-${EnvId}-${Component}-task'
          Essential: 'true'
          PortMappings:
            - ContainerPort: 80
              Protocol: tcp
          Image: !Sub '${AWS::AccountId}.dkr.ecr.eu-central-1.amazonaws.com/some-microservice:1.0.0
          Cpu: 256
          LogConfiguration:
            LogDriver: awslogs
            Options:
              awslogs-group: !ImportValue
                'Fn::Sub': 'stk-${EnvType}-${EnvId}-service-cluster-LogGroup'
              awslogs-region: !Ref 'AWS::Region'
              awslogs-stream-prefix: !Sub 'stk-${EnvType}-${EnvId}-${Component}-task'
  SomeMicroserviceService:
    Type: AWS::ECS::Service
    Properties:
      ServiceName: !Sub 'stk-${EnvType}-${EnvId}-${Component}-service'
      Cluster: !ImportValue
        'Fn::Sub': 'stk-${EnvType}-${EnvId}-service-cluster'
      DeploymentConfiguration:
        MaximumPercent: 200
        MinimumHealthyPercent: 100
      DesiredCount: 1
      TaskDefinition: !Ref SomeMicroserviceTask
      LoadBalancers:
      - TargetGroupArn: !Ref SomeMicroserviceTargetGroup
        ContainerPort: 80
        ContainerName: !Sub 'stk-${EnvType}-${EnvId}-${Component}-task'
  SomeMicroserviceTargetGroup:
    Type: AWS::ElasticLoadBalancingV2::TargetGroup
    Properties:
      Name: !Sub 'stk-${EnvType}-${EnvId}-${Component}-tg'
      Port: 80
      Protocol: HTTP
      TargetType: instance
      VpcId: !ImportValue
        'Fn::Sub': 'stk-${EnvType}-${EnvId}-VpcId'
      HealthCheckEnabled: true
      HealthCheckIntervalSeconds: 30
      HealthCheckPath: '/some-microservice/actuator/health'
      HealthCheckProtocol: HTTP
      HealthCheckTimeoutSeconds: 5
      HealthyThresholdCount: 5
      UnhealthyThresholdCount: 2
      Matcher:
        HttpCode: '200'
  SomeMicroserviceHttpListenerRule:
    Type: AWS::ElasticLoadBalancingV2::ListenerRule
    Properties:
      Priority: 1
      ListenerArn: !ImportValue
        'Fn::Sub': 'stk-${EnvType}-${EnvId}-load-balancer-http-listener-Arn'
      Actions:
        - Type: forward
          TargetGroupArn: !Ref SomeMicroserviceTargetGroup
      Conditions:
        - Field: 'path-pattern'
          PathPatternConfig:
            Values:
              - '/some-microservice/*'
  SomeMicroserviceHttpsListenerRule:
    Type: AWS::ElasticLoadBalancingV2::ListenerRule
    DependsOn:
      - SomeMicroserviceTargetGroup
    Properties:
      Priority: 1
      ListenerArn: !ImportValue
        'Fn::Sub': 'stk-${EnvType}-${EnvId}-load-balancer-https-listener-Arn'
      Actions:
        - Type: forward
          TargetGroupArn: !Ref SomeMicroserviceTargetGroup
      Conditions:
        - Field: 'path-pattern'
          PathPatternConfig:
            Values:
              - '/some-microservice/*'

EnvType, EnvId and Component are three parameters we always set in out templates. EnvType and EnvId are the same for the whole environment (ex. dev + 000). Component is template specific, set as a constant (ex. some-microservice).

There are a few caveats:

  • ALB does not cut the prefix of the path-pattern when forwarding to your microservice. So the request will be /some-microservice/some-endpoint and not just /some-endpoint.
  • Make sure your microservice provides a health check endpoint specified in HealthCheckPath. In the example above, the microservice has to implement /some-microservice/actuator/health health check endpoint.

I also had problems with services/tasks not working with ALB for these two reasons.

Once my microservice was simply serving /some-endpoint, not /some-microservice/some-endpoint and I was getting 404.

Then I've set a global prefix in the microservice and it stopped registering with ALB at all. After checking target groups I've found out that they do not have healthy targets. Apparently, I've forgotten to update the health check endpoint from /actuator/health to /some-microservice/actuator/health.

If there are problems I normally check the following:

  • ECS Service is OK and have running tasks
  • There's a target group with healthy targets
  • Load balancer has HTTP/HTTPS listeners
  • Listeners have registered rules with corresponding path patterns and forwarding to target groups
  • Security group configs are OK (SG for cluster accepts traffic from the load balancer SG)

Upvotes: 1

Mech
Mech

Reputation: 1504

If I understand your question correct you want to use same ALB for all listeners which points to their own target group.

This is possible directly using CloudFormation without using the custom resources.I have attached snippet of the template for reference

"PublicLoadBalancer": {
    "Type": "AWS::ElasticLoadBalancingV2::LoadBalancer",
    "Properties": {
        "Scheme": "internet-facing",
        "LoadBalancerAttributes": [
            {
                "Key": "idle_timeout.timeout_seconds",
                "Value": "30"
            }
        ],
        "Subnets": [
            {
                "Ref": "Subnet"
            },
            {
                "Ref": "Subnet2"
            }
        ],
        "SecurityGroups": [
            {
                "Ref": "ELBSecurityGroup"
            }
        ]
    }
},
"InitialTargetGroupPublic": {
    "Type": "AWS::ElasticLoadBalancingV2::TargetGroup",
    "Properties": {
        "HealthCheckIntervalSeconds": 5,
        "HealthCheckPath": "/login",
        "HealthCheckProtocol": "HTTP",
        "HealthCheckTimeoutSeconds": 3,
        "HealthyThresholdCount": 2,
        "Name": {
            "Fn::Join": [
                "-",
                [
                    {
                        "Ref": "AWS::StackName"
                    },
                    "tg-1"
                ]
            ]
        },
        "Port": 80,
        "Protocol": "HTTP",
        "UnhealthyThresholdCount": 2,
        "VpcId": {
            "Ref": "VPC"
        }
    }
},
"PublicLoadBalancerListener": {
    "Type": "AWS::ElasticLoadBalancingV2::Listener",
    "DependsOn": [
        "PublicLoadBalancer"
    ],
    "Properties": {
        "DefaultActions": [
            {
                "TargetGroupArn": {
                    "Ref": "InitialTargetGroupPublic"
                },
                "Type": "forward"
            }
        ],
        "LoadBalancerArn": {
            "Ref": "PublicLoadBalancer"
        },
        "Port": 80,
        "Protocol": "HTTP"
    }
},
"InitialTargetGroupPublic1": {
    "Type": "AWS::ElasticLoadBalancingV2::TargetGroup",
    "Properties": {
        "HealthCheckIntervalSeconds": 5,
        "HealthCheckPath": "/login",
        "HealthCheckProtocol": "HTTP",
        "HealthCheckTimeoutSeconds": 3,
        "HealthyThresholdCount": 2,
        "Name": {
            "Fn::Join": [
                "-",
                [
                    {
                        "Ref": "AWS::StackName"
                    },
                    "tg-2"
                ]
            ]
        },
        "Port": 50000,
        "Protocol": "HTTP",
        "UnhealthyThresholdCount": 2,
        "VpcId": {
            "Ref": "VPC"
        }
    }
},
"PublicLoadBalancerListener1": {
    "Type": "AWS::ElasticLoadBalancingV2::Listener",
    "DependsOn": [
        "PublicLoadBalancer"
    ],
    "Properties": {
        "DefaultActions": [
            {
                "TargetGroupArn": {
                    "Ref": "InitialTargetGroupPublic1"
                },
                "Type": "forward"
            }
        ],
        "LoadBalancerArn": {
            "Ref": "PublicLoadBalancer"
        },
        "Port": 50000,
        "Protocol": "HTTP"
    }
},

Upvotes: 0

Related Questions