Multi-arch docker build python arm64 Command '('lsb_release', '-a')' returned non-zero exit status 1

Question

For sake of this question, I have a hello world python script called script.py.

def hello_world():
    print("hello world")


if __name__ == "__main__":
    hello_world()

And have a corresponding Dockerfile:

FROM python:3.8-alpine

WORKDIR /app

COPY requirements.txt requirements.txt

RUN python3 -m pip install --upgrade pip
RUN python3 -m pip install -r requirements.txt

COPY . .

CMD [ "python3", "script.py"]

---

Scenario #1 (amd64 as host machine):

When I build for amd64 architecture on this host I am successful doing so.

$ docker buildx build --platform linux/amd64 -t sntshk/cotu:latest .
WARN[0000] No output specified for docker-container driver. Build result will only remain in the build cache. To push result image into registry use --push or to load image into docker use --load 
[+] Building 2.6s (11/11) FINISHED                                                                                          
 => [internal] load build definition from Dockerfile                                                                   0.0s
 => => transferring dockerfile: 277B                                                                                   0.0s
 => [internal] load .dockerignore                                                                                      0.0s
 => => transferring context: 2B                                                                                        0.0s
 => [internal] load metadata for docker.io/library/python:3.8-alpine                                                   2.4s
 => [auth] library/python:pull token for registry-1.docker.io                                                          0.0s
 => [1/6] FROM docker.io/library/python:3.8-alpine@sha256:9b3ca9c8e9ec086ea825227ac6d819e46794c29590f04d775d517a48ef6  0.0s
 => => resolve docker.io/library/python:3.8-alpine@sha256:9b3ca9c8e9ec086ea825227ac6d819e46794c29590f04d775d517a48ef6  0.0s
 => [internal] load build context                                                                                      0.1s
 => => transferring context: 7.55kB                                                                                    0.1s
 => CACHED [2/6] WORKDIR /app                                                                                          0.0s
 => CACHED [3/6] COPY requirements.txt requirements.txt                                                                0.0s
 => CACHED [4/6] RUN python3 -m pip install --upgrade pip                                                              0.0s
 => CACHED [5/6] RUN python3 -m pip install -r requirements.txt                                                        0.0s
 => CACHED [6/6] COPY . .

But the build fails for arm64 architecture:

[..output trimmed...]
#8 12.03   File "/usr/local/lib/python3.8/site-packages/pip/_vendor/distro.py", line 125, in linux_distribution
#8 12.03     return _distro.linux_distribution(full_distribution_name)
#8 12.03   File "/usr/local/lib/python3.8/site-packages/pip/_vendor/distro.py", line 681, in linux_distribution
#8 12.03     self.version(),
#8 12.03   File "/usr/local/lib/python3.8/site-packages/pip/_vendor/distro.py", line 741, in version
#8 12.03     self.lsb_release_attr('release'),
#8 12.03   File "/usr/local/lib/python3.8/site-packages/pip/_vendor/distro.py", line 903, in lsb_release_attr
#8 12.03     return self._lsb_release_info.get(attribute, '')
#8 12.03   File "/usr/local/lib/python3.8/site-packages/pip/_vendor/distro.py", line 556, in __get__
#8 12.03     ret = obj.__dict__[self._fname] = self._f(obj)
#8 12.03   File "/usr/local/lib/python3.8/site-packages/pip/_vendor/distro.py", line 1014, in _lsb_release_info
#8 12.03     stdout = subprocess.check_output(cmd, stderr=devnull)
#8 12.03   File "/usr/local/lib/python3.8/subprocess.py", line 415, in check_output
#8 12.03     return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
#8 12.03   File "/usr/local/lib/python3.8/subprocess.py", line 516, in run
#8 12.03     raise CalledProcessError(retcode, process.args,
#8 12.03 subprocess.CalledProcessError: Command '('lsb_release', '-a')' returned non-zero exit status 1.
------
Dockerfile:9
--------------------
   7 |     COPY requirements.txt requirements.txt
   8 |     
   9 | >>> RUN python3 -m pip install --upgrade pip
  10 |     RUN python3 -m pip install -r requirements.txt
  11 |     
--------------------
error: failed to solve: process "/dev/.buildkit_qemu_emulator /bin/sh -c python3 -m pip install --upgrade pip" did not complete successfully: exit code: 1

Scenario #2 (arm64 as host machine):

I won't put much noise here, but on arm64, building arm64 image has no issue. But the same error is seen when I try to build amd64 image on arm64 host.

---

My final question is, how do I build amd64 on a arm64 host?

Update 1:

I am running the build on arm64 for both arm64 and amd64.

docker buildx build --push --platform linux/amd64,linux/arm64 -t me/myimage:latest .

The scenario now is that I can only run arm64 image on arm machine, but on amd64 machine, I see this error:

standard_init_linux.go:228: exec user process caused: exec format error

I saw this answer on SO which talks about problem with build. What is wrong at the moment?

Answer 1

I appreciate Corralien for his initial answer which was great to get out of the lsb_release error.

Regarding the update in your question, please make sure the machine you are building on is capable of emulating the other type of architecture you are targeting.

Best way to know that is to check it in buildx builder.

$ docker buildx create --name multiarch
$ docker buildx use multiarch
$ docker buildx inspect --bootstrap
Name:   multiarch
Driver: docker-container

Nodes:
Name:      multiarch0
Endpoint:  unix:///var/run/docker.sock
Status:    running
Platforms: linux/arm64, linux/arm/v7, linux/arm/v6

Here you can see that Platforms listed are arm based only. And no matter whatever you pass --platform to buildx command, it won't yield a good result.

The solution

The package name might be different on different distros, but on Ubuntu it's called qemu-user-static.

$ apt install -y qemu-user-static

This will install QEMU emulators. If you check your builder now, you'd see more architectures.

$ docker buildx inspect --bootstrap
Name:   multiarch
Driver: docker-container

Nodes:
Name:      multiarch0
Endpoint:  unix:///var/run/docker.sock
Status:    running
Platforms: linux/arm64, linux/amd64, linux/riscv64, linux/ppc64le, linux/s390x, linux/386, linux/mips64le, linux/mips64, linux/arm/v7, linux/arm/v6

As a side note, you might have to restart your docker service in order to take effect.

I was able to rebuild and run amd64 on my amd64 machine.

$ docker run --rm sntshk/cotu
Unable to find image 'sntshk/cotu:latest' locally
[...output trimmed...]
Status: Downloaded newer image for sntshk/cotu:latest
standard_init_linux.go:228: exec user process caused: exec format error

$ docker rmi sntshk/cotu
[...output trimmed...]
Deleted: sha256:07d3c46c9599a88fa81c385391300ab877b98ef71b18cce942455025657edeb5

$ docker run --rm sntshk/cotu
Unable to find image 'sntshk/cotu:latest' locally
latest: Pulling from sntshk/cotu
[...output trimmed...]
Status: Downloaded newer image for sntshk/cotu:latest
hello world

Also have a look at Building Multi-Architecture Docker Images With Buildx by Artur Klauser.

Answer 2

Don't you need an arm image as suggested by the documentation:

Build multi-platform images

Finally, depending on your project, the language that you use may have good support for cross-compilation. In that case, multi-stage builds in Dockerfiles can be effectively used to build binaries for the platform specified with --platform using the native architecture of the build node. A list of build arguments like BUILDPLATFORM and TARGETPLATFORM is available automatically inside your Dockerfile and can be leveraged by the processes running as part of your build.

FROM --platform=$BUILDPLATFORM python:3.8-alpine AS build
...