Question Please help understand when CGO_ENABLED is must and unavoidable, and what happens with the go compilation process. When CGO_ENABLED is must After having read the articles in references , it looks as far as go has cross-compile native support for the target platform, CGO_ENABLED will never be a must. Is this correct? The situation when cgo and CGO_ENABLED is absolute-must is when go compiler cannot produce the binary code for the target platform. For instance, imagine I am writing a Space X control board embedded program in Go and I have to use C library and the compiler for the board. There is no other library or compiler available for the board. In such case, I have to do CGO_ENABLED=1 and tell cgo the compiler and the linker to use and where the C library binaries are copied in my laptop, and setup the compilers CFLAGS etc etc. Perhaps I would have to use C for all the coding instead of using Go in this case, but somehow I was in the situation that I had to use Go. This is when CGO_ENABLED=1 would be must. Is this correct? Otherwise, if especially go has the support for the target platform cross-compilation, using CGO_ENABLED=1 could be a short-cut to just re-use the existing C library for the target platform. Is this correct or is there any other reason why CGO_ENABLED is required? What happens I suppose when CGO_ENABLED=1 is used, basically there will be a composition of the binary from Go part, and the binary from the C library in the executable created by go build command. And at runtime, the execution goes back and forth between the two binary world of go and C. Go binary side would not know C side, so the go tools like debugger would not be available. I suppose whether the executable hard-links all the libraries or uses dynamic linking depends on the target platform compiler. Are these correct understandings? The reason CGO_ENABLED=0 by default for cross-compilation is because we should use the go built-in support of the target platform cross-compilation and there should be no reason not to do so. The reason CGO_ENABLED=1 by default for the native local platform is because the cgo compiler knows (or the author of the compiler knows) the local machine architecture and the OS libraries (or well known 3rd party's) available, hence best-optimization can be expected? But I am not sure this is true because the go compiler itself can be best-optimized for the local OS and architecture, then why need to use cgo? Kindly explain why CGO_ENABLED=1 by default for the native local platform. References cgo is not Go Some people, when confronted with a problem, think “I know, I’ll use cgo.” Now they have two problems. cgo is an amazing technology which allows Go programs to interoperate with C libraries. It’s a tremendously useful feature without which Go would not be in the position it is today. cgo is key to ability to run Go programs on Android and iOS. However, and to be clear these are my opinions, I am not speaking for anyone else, I think cgo is overused in Go projects. I believe that when faced with reimplementing a large piece of C code in Go, programmers choose instead to use cgo to wrap the library, believing that it is a more tractable problem. I believe this is a false economy. Obviously, there are some cases where cgo is unavoidable, most notably where you have to interoperate with a graphics driver or windowing system that is only available as a binary blob. But those cases where cgo’s use justifies its trade-offs are fewer and further between than many are prepared to admit. Go’s support for cross compilation is best in class. As of Go 1.5 you can cross compile from any supported platform to any other platform with the official installer available on the Go project website. By default cgo is disabled when cross compiling. Normally this isn’t a problem if your project is pure Go. When you mix in dependencies on C libraries, you either have to give up the option to cross compile your product, or you have to invest time in finding and maintaining cross compilation C toolchains for all your targets. The number of platforms that Go supports continues to grow. Go 1.5 added support for 64 bit ARM and PowerPC. Go 1.6 adds support for 64 bit MIPS, and IBM’s s390 architecture is touted for Go 1.7. RISC-V is in the pipeline. If your product relies on a C library, not only do you have the all problems of cross compilation described above, you also have to make sure the C code you depend on works reliably on the new platforms Go is supporting — and you have to do that with the limited debuggability a C/Go hybrid affords you. Which brings me to my next point. Why does CGO_ENABLE make a such impact on virtual memory? the compiler of the "reference" Go implementation (historically dubbed "gc"; that one, available for download from the main site) by default produces statically-linked binaries. This means, such binaries rely only on the so-called "system calls" provided by the OS kernel and do not depend on any shared libraries provided by the OS (or 3rd parties). On Linux-based platforms, this is not completely true: in the default setting (building on Linux for Linux, i.e., not cross-compiling) the generated binary is actually linked with libc and with libpthread (indirectly, via libc). cmd/cgo: document how CGO_ENABLED works today The cgo tool is enabled by default for native builds on systems where it is expected to work. It is disabled by default when cross-compiling. You can control this by setting the CGO_ENABLED environment variable when running the go tool: set it to 1 to enable the use of cgo, and to 0 to disable it. The go tool will set the build constraint "cgo" if cgo is enabled. When cross-compiling, you must specify a C cross-compiler for cgo to use. You can do this by setting the CC_FOR_TARGET environment variable when building the toolchain using make.bash, or by setting the CC environment variable any time you run the go tool. The CXX_FOR_TARGET and CXX environment variables work in a similar way for C++ code. C? Go? Cgo! cgo

Reputation: 22368

When using CGO_ENABLED is must and what happens

Question

Please help understand when CGO_ENABLED is must and unavoidable, and what happens with the go compilation process.

When CGO_ENABLED is must

After having read the articles in references, it looks as far as go has cross-compile native support for the target platform, CGO_ENABLED will never be a must. Is this correct?

The situation when cgo and CGO_ENABLED is absolute-must is when go compiler cannot produce the binary code for the target platform. For instance, imagine I am writing a Space X control board embedded program in Go and I have to use C library and the compiler for the board. There is no other library or compiler available for the board.

In such case, I have to do CGO_ENABLED=1 and tell cgo the compiler and the linker to use and where the C library binaries are copied in my laptop, and setup the compilers CFLAGS etc etc. Perhaps I would have to use C for all the coding instead of using Go in this case, but somehow I was in the situation that I had to use Go. This is when CGO_ENABLED=1 would be must. Is this correct?

Otherwise, if especially go has the support for the target platform cross-compilation, using CGO_ENABLED=1 could be a short-cut to just re-use the existing C library for the target platform. Is this correct or is there any other reason why CGO_ENABLED is required?

What happens

I suppose when CGO_ENABLED=1 is used, basically there will be a composition of the binary from Go part, and the binary from the C library in the executable created by go build command. And at runtime, the execution goes back and forth between the two binary world of go and C. Go binary side would not know C side, so the go tools like debugger would not be available.

I suppose whether the executable hard-links all the libraries or uses dynamic linking depends on the target platform compiler.

Are these correct understandings?

The reason CGO_ENABLED=0 by default for cross-compilation is because we should use the go built-in support of the target platform cross-compilation and there should be no reason not to do so.

The reason CGO_ENABLED=1 by default for the native local platform is because the cgo compiler knows (or the author of the compiler knows) the local machine architecture and the OS libraries (or well known 3rd party's) available, hence best-optimization can be expected? But I am not sure this is true because the go compiler itself can be best-optimized for the local OS and architecture, then why need to use cgo?

Kindly explain why CGO_ENABLED=1 by default for the native local platform.

References

cgo is not Go

Some people, when confronted with a problem, think “I know, I’ll use cgo.” Now they have two problems.

cgo is an amazing technology which allows Go programs to interoperate with C libraries. It’s a tremendously useful feature without which Go would not be in the position it is today. cgo is key to ability to run Go programs on Android and iOS.

However, and to be clear these are my opinions, I am not speaking for anyone else, I think cgo is overused in Go projects. I believe that when faced with reimplementing a large piece of C code in Go, programmers choose instead to use cgo to wrap the library, believing that it is a more tractable problem. I believe this is a false economy.

Obviously, there are some cases where cgo is unavoidable, most notably where you have to interoperate with a graphics driver or windowing system that is only available as a binary blob. But those cases where cgo’s use justifies its trade-offs are fewer and further between than many are prepared to admit.

Go’s support for cross compilation is best in class. As of Go 1.5 you can cross compile from any supported platform to any other platform with the official installer available on the Go project website.

By default cgo is disabled when cross compiling. Normally this isn’t a problem if your project is pure Go. When you mix in dependencies on C libraries, you either have to give up the option to cross compile your product, or you have to invest time in finding and maintaining cross compilation C toolchains for all your targets.

The number of platforms that Go supports continues to grow. Go 1.5 added support for 64 bit ARM and PowerPC. Go 1.6 adds support for 64 bit MIPS, and IBM’s s390 architecture is touted for Go 1.7. RISC-V is in the pipeline. If your product relies on a C library, not only do you have the all problems of cross compilation described above, you also have to make sure the C code you depend on works reliably on the new platforms Go is supporting — and you have to do that with the limited debuggability a C/Go hybrid affords you. Which brings me to my next point.
Why does CGO_ENABLE make a such impact on virtual memory?

the compiler of the "reference" Go implementation (historically dubbed "gc"; that one, available for download from the main site) by default produces statically-linked binaries. This means, such binaries rely only on the so-called "system calls" provided by the OS kernel and do not depend on any shared libraries provided by the OS (or 3rd parties).

On Linux-based platforms, this is not completely true: in the default setting (building on Linux for Linux, i.e., not cross-compiling) the generated binary is actually linked with libc and with libpthread (indirectly, via libc).
cmd/cgo: document how CGO_ENABLED works today

The cgo tool is enabled by default for native builds on systems where it is expected to work. It is disabled by default when cross-compiling. You can control this by setting the CGO_ENABLED environment variable when running the go tool: set it to 1 to enable the use of cgo, and to 0 to disable it. The go tool will set the build constraint "cgo" if cgo is enabled.

When cross-compiling, you must specify a C cross-compiler for cgo to use. You can do this by setting the CC_FOR_TARGET environment variable when building the toolchain using make.bash, or by setting the CC environment variable any time you run the go tool. The CXX_FOR_TARGET and CXX environment variables work in a similar way for C++ code.
C? Go? Cgo!
cgo

Upvotes: 70

Answers (2)

kurt

Reputation: 653

If you're running on an Alpine image, it is impossible to compile and run Go programs in Alpine images right away. You must disable CGO by setting the environment variable CGO_ENABLED to false (the default value is true).

You can do this either by:

Adding go env -w CGO_ENABLED=0 like Robert mentions in his comment or,
Setting the env value prefixing it in the go command, e.g.: CGO_ENABLED=0 GOARCH=amd64 GOOS=linux go build -ldflags '-s -w' -tags lambda.norpc -o bin/<YOUR_FUNC>/bootstrap <YOUR_PATH>/main.go

See: https://megamorf.gitlab.io/2019/09/08/alpine-go-builds-with-cgo-enabled/

Upvotes: 5

Martin Tournoij

Reputation: 27852

Many things are only available as C libraries, and re-implementing that all in Go would be costly. cgo has its downsides, but it can be a good trade-off. Even the standard library uses it (net for DNS lookups, os/user for user lookups) because it doesn't re-implement 100% of the behaviour in Go.

Cross-compiling C code is still rather hard; you'll need the target architecture's C compiler and toolchain (e.g. CC=aarch64-linux-musl-gccgo build to build an arm64 binary). None of that is installed by default so for most people cgo simply won't work when cross-compiling; they need to take manual steps to set it up first.

cgo often isn't strictly required (like in the net and os/user packages), so disabling it by default seems the most user-friendly option.

But there are no such constraints on the native platform, and it's expected to work by default without any user setup; so why not enable it by default?

Upvotes: 17