CODEWITHSUNDEEP
docker
Greg
Greg

Reputation: 11512

How can I fix 'No space left on device' error in Docker?

I'm running a Mac-native Docker (no virtualbox/docker-machine).

I have a huge image with a lot of infrastructure in it (Postgres, etc.). I have run cleanup scripts to get rid of a lot of cruft--unused images and so forth.

When I run my image I get an error like:

could not create directory "/var/lib/postgresql/data/pg_xlog": No space left on device

On my host Mac /var is sitting at 60% space available and generally my disk has lots of storage free.

Is this some Docker configuration I need to bump up to give it more resources?

Relevant lines from mount inside docker:

none on / type aufs (rw,relatime,si=5b19fc7476f7db86,dio,dirperm1)
/dev/vda1 on /data type ext4 (rw,relatime,data=ordered)
/dev/vda1 on /etc/resolv.conf type ext4 (rw,relatime,data=ordered)
/dev/vda1 on /etc/hostname type ext4 (rw,relatime,data=ordered)
/dev/vda1 on /etc/hosts type ext4 (rw,relatime,data=ordered)
/dev/vda1 on /var/lib/postgresql/data type ext4 (rw,relatime,data=ordered)

Here’s df:

[11:14]  
Filesystem     1K-blocks    Used Available Use% Mounted on
none           202054928 4333016 187269304   3% /
tmpfs            1022788       0   1022788   0% /dev
tmpfs            1022788       0   1022788   0% /sys/fs/cgroup
/dev/vda1      202054928 4333016 187269304   3% /data
shm                65536       4     65532   1% /dev/shm
tmpfs             204560     284    204276   1% /run/docker.sock

Upvotes: 63

Views: 68890

Answers (6)

mahendra150
mahendra150

Reputation: 1

docker system prune //this command past only, for clear cache etc

enter image description here

Upvotes: -3

Erkhembayar Gantulga
Erkhembayar Gantulga

Reputation: 109

This issue also happened to me.

For me, the fix was to increase the 'Virtual Disk Limit' size in Docker Desktop (Mac).

Virtual Disk Limit setting

Upvotes: 7

Yakob Ubaidi
Yakob Ubaidi

Reputation: 1962

if you want to prune everything use docker image prune -a

Upvotes: 0

Khateeb321
Khateeb321

Reputation: 2104

I ran into the same issue, running docker system prune --volumes resolved the problem.

"Volumes are not pruned by default, and you must specify the --volumes flag for docker system prune to prune volumes."

See: https://docs.docker.com/config/pruning/#prune-everything

Upvotes: 43

pcambra
pcambra

Reputation: 721

I haven't found many options for this, the main issue in github is https://github.com/docker/for-mac/issues/371

Some of the options suggested there are:

  • If you can remove all images/containers, you can follow these instructions:

docker rm $(docker ps -a -q) docker rmi $(docker images -q) docker volume rm $(docker volume ls |awk '{print $2}') rm -rf ~/Library/Containers/com.docker.docker/Data/*

  • You can try to prune all unused images/containers but this has proven not very effective:

docker system prune

See also: How do you get around the size limitation of Docker.qcow2 in the Docker for Mac? And https://forums.docker.com/t/no-space-left-on-device-error/10894/26

Upvotes: 42

avivr
avivr

Reputation: 1523

I ran into this recently on with a docker installation on linux that uses the devicemapper storage driver (default). There was indeed a docker configuration I needed to change to fix this.

Docker images are made of read-only layers of filesystem snapshots, each layer created by a command in your Dockerfile, which are built on top of a common base storage snapshot. The base snapshot is shared by all your images and has a file system with a default size of 10GB. When you run your image you get a new writable layer on top of all the layers in the image, so you can add new files in your running container but it's still eventually based on the same base snapshot with the 10GB filesystem. This is at least true for devicemapper, not sure about other drivers. Here is the relevant documentation from docker.

To change this default value to something else, there's a daemon parameter you can set, e.g. docker daemon --storage-opt dm.basesize=100G. Since you probably don't run the daemon manually need to edit the docker daemon options in some file, depending on how you run the docker daemon. With docker for mac you can edit the daemon parameters as JSON in the preferences under Daemon->Advanced. You probably need to add something like this:

{
    "storage-opts": ["dm.basesize=100G"]
}

(but like I said, I had this problem on linux, so didn't try the above).

Anyway in order for this to take effect, you'll need to remove all your existing images (so that they're re-created on top of the new base snapshot with the new size). See storage driver options.

Upvotes: 7

Related Questions