CODEWITHSUNDEEP
dockerdocker-composeamazon-ecs
Peter
Peter

Reputation: 2394

Docker compose and Fargate rollbacks

I'm trying to improve my service by setting a rollback strategy in case my changes crash the container and keep the tasks exiting.

Context: I have a simple service that I update changing the tag.

services
  web:
    image: AWS-Account-Id.dkr.ecr.us-east-1.amazonaws.com/my-service:1
    environment:
      - ENV=dev
    ports: ["80:80"]

I make some change in the docker image, build, tag, and push it to ECR. Then update the tag to 2 (for example) and run docker compose up.

Let's say that I introduce an error and the container starts but then stops (due to the error) it will keep constantly trying to start and stop the container with error: Essential container in task exited

Is there a way in docker-compose to set a condition where if it tries to start the container web 2 times and the tasks fail to reach and maintain the status of running, rollback changes or do a cloudformation cancel update operation?

There is a load balancer that listens to port 80 and I also added a health check to the

    healthcheck:
      test: ["CMD", "curl", "-f", "/status"]
      interval: 1m
      timeout: 10s
      retries: 2
      start_period: 40s

But I cannot make it work. Tasks keep exiting and the cloudformation deployment keeps going.

Upvotes: 0

Views: 428

Answers (1)

xeon
xeon

Reputation: 954

This is no direct way for this, but you can consider this approach:

  1. Create a Wait Condition and a WaitCondition Handle Resource.
  2. Calibrate how long it usually takes for the Task / Container to start and set the timeout accordingly.
  3. Configure the application to post a success signal to the endpoint URL on successful setup.
  4. Ensure that the Service and the waitcondition handle start updating / creating in parallel.

If the time exceeds the timeout period, the wait condition handle will rollback.

Thing to consider: On every operation the waitcondition handle and the waitcondition resources will need to be re-created. Easy way to do that is to modify the logical id of the resources. There can be a parameters / template hash calculator that will add the hash as suffix to the wait condition resources. Thus, if there's a change in the parameters / template, the wait condition resources will be recreated automatically.

Upvotes: 2

Related Questions