CODEWITHSUNDEEP
dockerkubernetes
JiaMing Lin
JiaMing Lin

Reputation: 1082

A memory issue, how to manage the Exited Docker Containers?

I am now doing experiments on using Kubernetes and Docker to provisioning services such as WordPress, Spark and Storm on 10 physical machines

But after lots times of launching and terminating Docker Containers, the used memory would increase even I kill all the Containers via Kubernetes delete or Docker kill commands.

I noticed that there were lots of Containers with status Exited, and after I remove all the Exited Containers, it frees lots of memories.

And I came up with a solution, which is writing a removing exited containers schedule into cron table on each Docker host.

But is this appropriate? if not, how can I release the memories?

Upvotes: 1

Views: 1200

Answers (2)

Yu-Ju Hong
Yu-Ju Hong

Reputation: 7287

It is not recommended to use external container garbage collection scripts. Kubernetes relies on exited containers as tombstones to reconstruct the pod status and/or serve logs. Even if you don't care about container logs, if you remove the exited containers before kubernetes examines them and properly records the status, it may cause inaccurate status and restart decisions. This reliance may be eliminated in the future.

For now, the best way to achieve more aggressive container garbage collection is through adjusting the parameters, as detailed in this guide.

FYI, there are also open issues to improve the garbage collection behavior. #13287 is one example.

If you really want to clean up the containers yourself, it is safe to remove containers associated with deleted pods. Removing multiple exited containers that belong to the same pod/container while keeping the most recent few exited containers is also relatively low-risk.

Upvotes: 4

VonC
VonC

Reputation: 1323263

It is appropriate, if you have no intermediate state to save (docker commit).

Another similar solution is to docker run with the --rm option: if it exits, the container will be removed automatically.
But if you have to debug errors when the container fail to start, you would need to docker run without --rm in that particular case.

Upvotes: 1

Related Questions