CODEWITHSUNDEEP
amazon-web-servicesdockeramazon-ec2aws-opsworks
Sathed
Sathed

Reputation: 846

Where does docker store it's temp files during extraction?

Before you chastise me for having a 12 GB image, know that it's the only way we can handle this specific scenario and still automate the build process. When I'm done with this project, I'll put myself in timeout - I promise.

Now, I'm working on a project with Opsworks and I need to pull down that very large image I mentioned. However, unless we use the AWS console, we can't change the size of the root volume on the EC2 instance in Opsworks. So, I mount a 50 GB volume at /var/lib/docker/ at startup. However, the Docker API is still saying that I'm running out of disk space (50 GB is much more than is necessary) during the image extraction. I'm guessing that docker is storing these temporary files somewhere other than in /var/lib/docker/ but for the life of me, I can't figure out where.

Hell, if someone can come up with a way to increase the root volume size of an Opsworks EC2 instance without creating a custom image, that would be nice too.

Any help would be appreciated.

Upvotes: 4

Views: 11984

Answers (3)

Sohail Si
Sohail Si

Reputation: 2976

To change docker's temp directory:

  1. sudo systemctl cat docker.service
  2. sudo systemctl show docker.service and look for --data-root
    • sudo systemctl show docker.service |grep ExecStart
  3. systemctl show --property=Environment docker
  4. docker info

How to change it: You first need to stop the docker service, and then

sudo systemctl edit docker.service

Then add ( don't forget to replace YOURNEWPATH below )

[Service]
ExecStart=
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --data-root=YOURNEWPATH

Then reload, then verify, then start again, etc.

See steps below for the bigger picture:


set -exu

SERVICENAME="docker"
SERVICENAME_EXTRA="docker.socket"
NEW_TEMP="/mnt/volume_lon1_01/dockerd1"

echo "This script is never tested. You too, don't run it. It is for documentation purposes only."
exit 1

# keep a backup somewhere
sudo systemctl cat docker.service >$(mktemp  --suffix=.docker-service-config-values-backup.txt -p .)

# Stop docker and related services
sudo systemctl stop $SERVICENAME_EXTRA
sudo systemctl stop $SERVICENAME


# {
sudo mkdir -p /etc/systemd/system/$SERVICENAME.service.d/

sudo tee /etc/systemd/system/$SERVICENAME.service.d/override.conf \
<< EOF_OVERRIDE.CONF

[Service]
# Added by Looking up from StackOverflow44010124
Environment="DOCKER_TMP=$NEW_TEMP"
ExecStart=
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock --data-root=$NEW_TEMP

EOF_OVERRIDE.CONF

# or manually:
# sudo systemctl edit docker.service

sudo mkdir $NEW_TEMP

# }

# Reloads the config, necessary before the start
sudo systemctl daemon-reload

# Verify changes by eye:
docker info
sudo systemctl cat docker.service
systemctl show --property=Environment $SERVICENAME

sudo systemctl start $SERVICENAME
sudo systemctl start $SERVICENAME_EXTRA

echo 'Now you can delete the old temp, traditionally in /var/lib/docker/*'
  • Caution: Keep a note of previous values
  • As of Docker version 24.0.5 build 24.0.5-0ubuntu1~22.04.1.
  • The above script is for documentation of steps, and has not been tested as a whole script.
  • Delete the old ones. Check space using ncdu /var/lib/docker/, ncdu /, df -h, etc.

Upvotes: 0

James Bradbury
James Bradbury

Reputation: 1758

I'm using Docker on Ubuntu and to needed to move the temporary directory to another drive to avoid thrashing the USB disk.

The following method worked for me.

  1. Stop docker
  2. Move the directory and contents.
  3. Create a symlink from the original location to the new one.
  4. Start docker.

Upvotes: -1

programmerq
programmerq

Reputation: 6534

By default, Docker will use /var/lib/docker/tmp for it's temporary directory. This can be overridden with the DOCKER_TMP environment variable for the docker daemon.

Upvotes: 4

Related Questions