CODEWITHSUNDEEP
amazon-ec2amazon-ecs
user8479984
user8479984

Reputation: 501

Does ECS update-service command marks the container instance state to draining when use with the --force-new-deployment option?

The command:

aws ecs update-service --service my-http-service --task-definition amazon-ecs-sample --force-new-deployment

As per AWS docs: You can use this option (--force-new-deployment) to trigger a new deployment with no service definition changes. For example, you can update a service's tasks to use a newer Docker image with the same image/tag combination (my_image:latest ) or to roll Fargate tasks onto a newer platform version.

My question, if I use '--force-new-deployment' (as I will use the exisiting tag or definition), will the underline 'ECS Instance' automatically set to DRAINING state, so that any new task (if any) will not start in the EXISTING ecs-instance that is suppose to go away during rolling-update deployment strategy (or deployment controller) ?

In other words, will there be any chance:

  1. For a new task to be created on the existing/old container instance, that is suppose to go away during rolling update.

  2. Also, what would happen with the ongoing task that is running on this existing/old container instance, that is suppose to go away during rolling update.

Ref: https://docs.aws.amazon.com/cli/latest/reference/ecs/update-service.html

Upvotes: 1

Views: 2634

Answers (1)

shariqmaws
shariqmaws

Reputation: 8890

Please note that no Container instance is going anywhere with this 'update-service' command. This command will only create a new Deployment under the ECS service and when the new tasks become healthy, remove the old task(s).

Edit 1:

How about the request that were served by old task?

I am assuming the tasks are behind an Application Load Balancer. In this case, old tasks will be deregistered from the ALB.

Note: In the following discussion, target is the ECS Task.

To give you a brief description on how the Deregistration Delay works with ECS, the following is the sequential order when a task connected to an ALB is stopped. It can be due to a scale in event, deployment of a new task definition, decrease of the number of tasks, force deployment, etc.

  1. ECS sends DeregisterTargets call and the targets change the status to "draining". New connections will not be served to these targets.

  2. If the deregistration delay time elapsed and there are still in-flight requests, the ALB will terminate them and clients will receive 5XX responses originated from the ALB.

  3. The targets are deregistered from the target group.

  4. ECS will send the stop call to the tasks and the ECS-agent will gracefully stop the containers (SIGTERM).

  5. If the containers are not stopped within the stop timeout period (ECS_CONTAINER_STOP_TIMEOUT by default 30s) they will force stopped (SIGKILL).

As per the ELB documentation [1] if a deregistering target has no in-flight requests and no active connections, Elastic Load Balancing immediately completes the deregistration process, without waiting for the deregistration delay to elapse. However, even though target deregistration is complete, the status of the target will be displayed as draining and you can see the registered Target of the old task is still present in the TargetGroup console with status as draining until the deregistration delay elapses.

The design of ECS is to stop the Task after the completion of Draining process as mentioned in the ECS document [2] and hence the ECS Service waits for the TargetGroup to complete the Draining process before issuing the stop call.

Ref:

[1] Target Groups for Your Application Load Balancers - Deregistration Delay - https://docs.aws.amazon.com/elasticloadbalancing/latest/application/load-balancer-target-groups.html#deregistration-delay

[2] Updating a Service - https://docs.aws.amazon.com/AmazonECS/latest/developerguide/update-service.html

Upvotes: 2

Related Questions