Compiling go library without GCO to run on alpine, error in libczmq

Question

When trying to run my binary on alpine, I got the error:

... binary not found

which typically happens when there's a problem with architecture, or as I found, glibc. I searched and discovered that alpine instead uses muslc, an alternative C library. I then found this Installed Go binary not found in path on Alpine Linux Docker that teaches how to compile without CGO, which is the thing that permits loading C libraries from go:

GOOS=linux GOARCH=amd64 CGO_ENABLED=0 go build -o [name of binary]

When I run this, I get:

go build gopkg.in/zeromq/goczmq.v4: no buildable Go source files in /home/lucas/Go/src/gopkg.in/zeromq/goczmq.v4

I suspect that it's because libczmq is just a wrapper for the C written libzmq. In this case, how do I compile in a way that I can use libczmq? Why exactly CGO must be disabled in alpine?

What's exactly CGO? Shouldn't it use libc when available but muslc when not? I'd like to know more about what's happening in the background.

Note: I'm compiling outside alpine, in ubuntu, is that a problem?

Answer 1

I know this question is about three years old now but there is nothing on the Internet about this and I was facing the exact same problem and it took about two days before I finally found a proper solution.

In short, I had a Go project that uses goczmq that I wanted to compile to a complete binary to put it into a FROM scratch Docker container (although in this case, an alpine container would work just as well). On the Internet, people now tend to tell you to set CGO_ENABLED=0 and everything will work fine, which is true. This tells Go to not use CGO, which allows you to use C libraries in your Go code, but needs those C libraries to be available on the system when you run your code. As you have figured out, alpine does not have those C libraries (or, well, it has different ones in muslc instead of glibc).

However, in our case, this is not helpful. We want our Go code to be able to use existing C libraries because we use goczmq, which, as you have identified, is a Go wrapper for czmq which itself is a C wrapper for libzmq (which is written in C++, which makes our life even harder).

I solved this problem by using a static binary. Instead of linking to whatever C libraries are available on the target system (in this case muslc on alpine) dynamically, I compile my code AND all the libraries (muslc, czmq and libzmq) into one unified binary.

I'm using Docker multi-stage builds based on alpine here but in theory you could also do this directly on your computer.

# stage 1: build the binary
# we are using alpine Linux with the latest version of golang
FROM golang:1.13-alpine as golang

# first install some dependencies
# (we are using the static versions for each for them as we want these libraries included statically, not dynamically!)
# czmq requires libzmq which in turn requires libsodium
# on alpine Linux we also need to install some specific tools to build C and C++ programs
# libsodium also requires libuuid, which is included in util-linux-dev
RUN apk add --no-cache libzmq-static czmq-dev libsodium-static build-base util-linux-dev

# now we do the magic command mentioned here
# https://stackoverflow.com/questions/34729748/installed-go-binary-not-found-in-path-on-alpine-linux-docker?noredirect=1&lq=1
# this fools the C compiler into thinking we have glibc installed while we are actually using musl
# since we are compiling statically, it makes sense to use musl as it is smaller
# (and it uses the more permissive MIT license if you want to distribute your binary in some form, but check your other libraries before!)
RUN mkdir /lib64 && ln -s /lib/libc.musl-x86_64.so.1 /lib64/ld-linux-x86-64.so.2

# create your project directory for the Go project
WORKDIR /go/src/github.com/<username>/<projectname>/

# copy in all your Go files, assuming these are in the same directory as your Dockerfile
COPY . .

# here is the first hack: we need to tell CGO to use g++ instead of gcc or else it will struggle with libzmq, which is written in C++
# creating and empty C++ file actually works for this
RUN touch ./dummy.cc

# now run go install (go build could also work here but your binary would end up in a different directory)
# the -ldflags will be passed along the CGO toolchain
# -extldflags is especially important here, it has two important flags that we need:
# -static tells the compiler to use static linking, which does the actual magic of putting everything into one binary
# -luuid is needed to correctly find the uuid library that czmq uses
RUN go install -a -ldflags '-linkmode external -w -s -extldflags "-static -luuid" ' .

# stage 2: here is your actual image that will later run on your Docker host
# you can also use alpine here if you so choose
FROM scratch

# now we just copy over the completed binary from the old builder image
COPY --from=golang /go/bin/<projectname> bin

# and we start our program
ENTRYPOINT ["./bin"]

Now this almost works! The only thing left is to add this statement to the beginning of your main.go file or else CGO is confused about what you are doing:

import "C"

This is important even if you don't directly use CGO in your program.