I can't install python packages by using pip3 on alpine

Question

Could not install packages due to an EnvironmentError: [Errno 39] Directory not empty: '/tmp/pip-install-xo15v1tw/mock/mock'

I created an alpine container. Dockerfile is

FROM alpine:latest

I built and run it.

#sudo docker build -t alpine .
#sudo docker run -it alpine

Then, I installed python3

#apk add python3

Then, I tried to install mock package and I got error.

#pip3 install mock
...
Could not install packages due to an EnvironmentError: [Errno 39] Directory not empty: '/tmp/pip-install-xo15v1tw/mock/mock'

I tried to install other python packages, but I always got the same error. I'm running on the container on WSL. The version of alpine is 3.9.4.

How do I solve this problem?

Answer 1

I had the exact same issue running Docker (version 17.09.0 in Ubuntu 16.04, and version 18.06.1 in Ubuntu 18.04) in WSL on Windows 1903, build 18362.145 (my test example is below).

The problem appears to be related to the default overlay2 (and also affects the older overlay) storage driver. Changing to the vfs storage driver by adding the following to /etc/docker/daemon.json makes pip work correctly:

{
  "storage-driver": "vfs"
}

However, the vfs storage driver is not recommended due to it's poor performance and high disk usage (it creates a complete copy of the file system in each layer). This is particularly relevant for docker builds where a complete copy of the file system is made for each line in the Dockerfile. So I'm not sure if this is a solution, but it at least indicates where the problem comes from. No other storage drivers appear to be compatible with WSL.

Following are two completely different situations where the overlay2/overlay storage driver appears to cause errors that do not appear to be related to storage at all. I'm sure this would also cause many other errors. I'm not sure if this was an issue before Windows 1903 (I didn't try docker in WSL then), and I'm also not sure what could be causing this (is it on the WSL side, or the overlay2/overlay side? - maybe someone with more knowledge than me would have a comment on this).

---

My test when using the overlay2/overlay storage driver is as follows:

$ docker run -it alpine
/ # apk add --update py3-pip
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz
(1/11) Installing libbz2 (1.0.6-r6)
(2/11) Installing expat (2.2.6-r0)
(3/11) Installing libffi (3.2.1-r6)
(4/11) Installing gdbm (1.13-r1)
(5/11) Installing xz-libs (5.2.4-r0)
(6/11) Installing ncurses-terminfo-base (6.1_p20190105-r0)
(7/11) Installing ncurses-terminfo (6.1_p20190105-r0)
(8/11) Installing ncurses-libs (6.1_p20190105-r0)
(9/11) Installing readline (7.0.003-r1)
(10/11) Installing sqlite-libs (3.28.0-r0)
(11/11) Installing python3 (3.6.8-r2)
Executing busybox-1.29.3-r10.trigger
OK: 68 MiB in 25 packages
/ # pip3 install mock
Collecting mock
  Downloading https://files.pythonhosted.org/packages/05/d2/f94e68be6b17f46d2c353564da56e6fb89ef09faeeff3313a046cb810ca9/mock-3.0.5-py2.py3-none-any.whl
Could not install packages due to an EnvironmentError: [Errno 39] Directory not empty: '/tmp/pip-install-btvxatk0/mock/mock'

/ #

When using the vfs storage driver:

$ docker run -it alpine
/ # apk add --update py3-pip
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz
(1/11) Installing libbz2 (1.0.6-r6)
(2/11) Installing expat (2.2.6-r0)
(3/11) Installing libffi (3.2.1-r6)
(4/11) Installing gdbm (1.13-r1)
(5/11) Installing xz-libs (5.2.4-r0)
(6/11) Installing ncurses-terminfo-base (6.1_p20190105-r0)
(7/11) Installing ncurses-terminfo (6.1_p20190105-r0)
(8/11) Installing ncurses-libs (6.1_p20190105-r0)
(9/11) Installing readline (7.0.003-r1)
(10/11) Installing sqlite-libs (3.28.0-r0)
(11/11) Installing python3 (3.6.8-r2)
Executing busybox-1.29.3-r10.trigger
OK: 68 MiB in 25 packages
/ # pip3 install mock
Collecting mock
  Downloading https://files.pythonhosted.org/packages/05/d2/f94e68be6b17f46d2c353564da56e6fb89ef09faeeff3313a046cb810ca9/mock-3.0.5-py2.py3-none-any.whl
Collecting six (from mock)
  Downloading https://files.pythonhosted.org/packages/73/fb/00a976f728d0d1fecfe898238ce23f502a721c0ac0ecfedb80e0d88c64e9/six-1.12.0-py2.py3-none-any.whl
Installing collected packages: six, mock
Successfully installed mock-3.0.5 six-1.12.0
You are using pip version 18.1, however version 19.1.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
/ #

---

I also found that running apt update in the ububtu official image, I got the following output when using the overlay2/overlay storage driver:

$ docker run -it ubuntu
root@d716ab2aee5c:/# apt update
Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic InRelease [242 kB]
Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
Err:1 http://security.ubuntu.com/ubuntu bionic-security InRelease
  At least one invalid signature was encountered.
Err:2 http://archive.ubuntu.com/ubuntu bionic InRelease
  At least one invalid signature was encountered.
Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]
Err:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease
  At least one invalid signature was encountered.
Err:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease
  At least one invalid signature was encountered.
Reading package lists... Done
W: GPG error: http://security.ubuntu.com/ubuntu bionic-security InRelease: At least one invalid signature was encountered.
E: The repository 'http://security.ubuntu.com/ubuntu bionic-security InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
W: GPG error: http://archive.ubuntu.com/ubuntu bionic InRelease: At least one invalid signature was encountered.
E: The repository 'http://archive.ubuntu.com/ubuntu bionic InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
W: GPG error: http://archive.ubuntu.com/ubuntu bionic-updates InRelease: At least one invalid signature was encountered.
E: The repository 'http://archive.ubuntu.com/ubuntu bionic-updates InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
W: GPG error: http://archive.ubuntu.com/ubuntu bionic-backports InRelease: At least one invalid signature was encountered.
E: The repository 'http://archive.ubuntu.com/ubuntu bionic-backports InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
root@d716ab2aee5c:/#

When using the vfs storage driver:

$ docker run -it ubuntu
root@1422a0121aa2:/# apt update
Get:1 http://archive.ubuntu.com/ubuntu bionic InRelease [242 kB]
Get:2 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
Get:4 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]
Get:5 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [717 kB]
Get:6 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [5436 B]
Get:7 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [535 kB]
Get:8 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages [1344 kB]
Get:9 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [4169 B]
Get:10 http://archive.ubuntu.com/ubuntu bionic/restricted amd64 Packages [13.5 kB]
Get:11 http://archive.ubuntu.com/ubuntu bionic/universe amd64 Packages [11.3 MB]
Get:12 http://archive.ubuntu.com/ubuntu bionic/multiverse amd64 Packages [186 kB]
Get:13 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1220 kB]
Get:14 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [10.8 kB]
Get:15 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [7239 B]
Get:16 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [837 kB]
Get:17 http://archive.ubuntu.com/ubuntu bionic-backports/universe amd64 Packages [3902 B]
Get:18 http://archive.ubuntu.com/ubuntu bionic-backports/main amd64 Packages [2496 B]
Fetched 16.7 MB in 26s (641 kB/s)
Reading package lists... Done
Building dependency tree
Reading state information... Done
12 packages can be upgraded. Run 'apt list --upgradable' to see them.
root@1422a0121aa2:/#