CODEWITHSUNDEEP
amazon-web-servicesdockercontainersamazon-ecs
riccio
riccio

Reputation: 87

AWS ECS task with no essential containers

I have an application that runs as a group of containers in an ECS task. Each container processes records from a kinesis shard. If there are n-shards then n-containers will be part of this task. Each container processes it's shard until it reaches the "end of the stream". At that point in time the container writes a result and exits. When all the containers exit the task is complete and a lambda processes the results.

My problem is that ECS requires that I designate at least one of the containers as "Essential". If the essential container finishes before the other containers finish their processing, then the entire task completes and all the incomplete containers exit.

Is there a way I can avoid designating any of the containers as 'essential'? I would like for them to complete their work without any regard to what the other containers in the task are doing.

Upvotes: 3

Views: 1085

Answers (1)

riccio
riccio

Reputation: 87

I ended up resolving this issue by adding another container to the task that will monitor the state of the non-essential containers.

Using this code allows me to see the state of the other containers.

import requests
import os
import time

uri = os.getenv('ECS_CONTAINER_METADATA_URI')

r = requests.get(uri + '/task')
data = r.json()

status_map = {}
for container in data['Containers']:
  status_map[container['Name']] = container['KnownStatus']

This allows you to check the state of the other containers and end the task when you are ready.

Upvotes: 3

Related Questions