CODEWITHSUNDEEP
amazon-web-servicesterraformamazon-ecs
bitbrain
bitbrain

Reputation: 619

How to keep the last X ECS task definitions active?

I have the following Terraform code to update a service with a new task definition:

resource "aws_ecs_task_definition" "app_definition" {
  family = "my-family"

  container_definitions = "${data.template_file.task_definition.rendered}"
  network_mode          = "bridge"
}

resource "aws_ecs_service" "app_service" {
  name            = "my-service"
  cluster         = "my-cluster"
  task_definition = "${aws_ecs_task_definition.app_definition.arn}"
  desired_count   = "1"
  iam_role        = "my-iam-role"
}

When updating my service, the last revision of my task definition becomes inactive. As a result, I can not select it when trying to manually roll back to a previous revision in the ECS console:

Error: No active task definition found

Ideally, I want to keep the last X revisions active so I can always manually roll back via the console if something goes wrong.

How can I achieve that?

Upvotes: 6

Views: 6991

Answers (4)

kk1957
kk1957

Reputation: 8824

Setting

skip_destroy = true

works now. Your original task definition has to be created with this setting. Then if you make any change to task definition, it will leave the previous task active, even though Terraform plan might suggest that previous revision will be destroyed.

Upvotes: 1

Meli
Meli

Reputation: 11

If you add skip_destroy to the task definition resource it will keep the previous version in the AWS console. I just ran into this issue and it actually worked.

  resource "aws_ecs_task_definition" "ecs_task_definition" {
    family                   = var.task_family
    container_definitions    = var.container_definitions
    skip_destroy             = true
}

Add "skip_destroy = true" and terraform apply first before making any changes to the task definition.

Note if you have this connected to a service with Terraform it will update to the latest version.

Upvotes: 1

bitbrain
bitbrain

Reputation: 619

A very simple approach is to hook into the lifecycle of Terraform:

resource "aws_ecs_task_definition" "app_definition" {
  family = "my-family"

  container_definitions = "${data.template_file.task_definition.rendered}"
  network_mode          = "bridge"

  # make sure Terraform does not unregister the task definition
  lifecycle {
    prevent_destroy = true
  }
}

As discussed in this Pull Request it prevents the destruction of the old task definition, thus keeps all task definitions active.

Upvotes: -5

ydaetskcoR
ydaetskcoR

Reputation: 56869

Terraform doesn't currently allow for this and its resource lifecycle model means that when you replace something (task definitions are immutable) Terraform must create a new one and destroy the old one.

With ECS task definitions also can't really be destroyed and instead are just marked as inactive as there may be tasks currently deployed that are using it until they are updated by the service to the new task definition.

There's 2 common ways of dealing with this and the need to be able to roll back to a previous version of a task definition.

The first is simply not to use Terraform to manage the task definition beyond initial creation and use something like the AWS ECS CLI tool to do this instead.

The other option, and the one that I use, is to have my CI (Gitlab CI in our case) generate a Docker image tagged with the commit SHA of the application to be deployed and then Terraform updates the task definition to the new commit SHA tagged image on an apply as well as updating the ECS service with the new task definition ARN.

When we want to roll back we use our CI's ability to roll back to a different commit, launching just the deploy job with the old commit SHA and so deploying the old image.

This keeps Terraform pretty agnostic of what's being deployed and makes the CI system responsible for deploying the required version which is normally latest but sometimes a specific commit if we have a manual click to deploy and of course the target previous version when rolling back.

It does mean that you can't launch roll backs through the AWS console but I actually like this as I want the CI system to be the source of truth for what is deployed at any time.

Upvotes: 9

Related Questions