Using CGO in Go? These 7 questions you should pay attention to!

Hello everyone, I am fried fish.

Today I will share with you the Cgo is not Go in the Go proverb. The original article has the same name and is slightly modified. The author is @Dave Cheney. The following "I" refer to the original author.

To borrow a quote from JWZ: some people face a problem and think "I know, I'll use cgo (to solve)".

After using cgo, they encountered two new problems.

What is Cgo

Cgo is an amazing technology that allows Go programs to interoperate with C libraries, which is a very useful feature.

Without it, Go wouldn't be where it is today. cgo is the key to running Go programs on Android and iOS.

overused

I personally think cgo is overused in Go projects, and when faced with reimplementing a large chunk of C code in Go, programmers choose to use cgo to wrap the library, thinking it's an easier problem to solve. But I think this is a wrong choice behavior.

Obviously cgo is unavoidable in some cases, most notably you have to interoperate with graphics drivers or windowing systems, which are only available as binary blobs. In these scenarios, the use of cgo justifies its tradeoffs with far fewer tradeoffs than many are prepared to admit.

The following is an incomplete list of tradeoffs you may not be aware of when building your Go project on the cgo library.

You need to think about this.

longer build times

When you import "C" in a Go package, go build needs to do more work to build your code.

Instead of simply passing a list of all .go files in scope to one invocation of go tool compilation, building your package contains the following work items:

• The cgo tool needs to be invoked to generate C to Go and Go to C related code.
• The C compiler on the system makes the call processing for each C file in the package.
• The various compilation units are combined into a single .o file.
• The generated .o file will go through the system's linker to correct the shared objects it references.

All of this work happens every time you compile or test your package, which happens quite often if you're actively working on that package.

The Go tools will parallelize this work where possible (including a full rebuild of all C code), and the compile time of the package will increase, and with it.

You also need to debug your C code on major platforms to avoid compilation failures due to compatibility.

complex build

One of Go's goals is to produce a language whose build process is self-describing; your program's source code contains enough information for a tool to build the project. That's not to say that using Makefiles to automate your build workflow is bad, but before cgo was introduced into projects, you probably didn't need anything other than the go tool to build and test.

After the introduction of cgo, you need to set all the environment variables to keep track of shared objects and header files that may be installed in strange places.

Also note that Go supports many platforms, while cgo does not. So you have to spend some time coming up with a solution for your Windows users.

Now your users must install the C compiler, not just the Go compiler. They also have to install the C library that your project depends on, and you'll also bear the cost of this technical support.

Cross compilation thrown out the window

Go's cross-compilation support is best-in-class. Starting with Go 1.5, you can cross-compile from any platform to any other platform via the official installer support on the Go project website.

By default, cgo is disabled when cross-compiling. Usually, if your project is pure Go, this is not a problem.

When you mix in dependencies on C libraries, you either give up on cross-compiling your code, or you have to invest the time to find and maintain a cross-compiled C toolchain for all targets in order to cross-compile.

The number of platforms that Go supports is constantly growing. Go 1.5 added support for 64-bit ARM and PowerPC. Go 1.6 added support for 64-bit MIPS, and IBM's s390 architecture was touted as Go 1.7. RISC-V is under development.

If your product depends on C libraries, not only do you have all of the above problems with cross-compilation, you also have to make sure that the C code you depend on works reliably on new platforms that Go supports -- and you have to do it in C/Go Do this with the limited debugging capabilities that mixed languages give you.

you lose access to all tools

Go has great tools; we have a race detector, pprof for profiling code, coverage, fuzzing, and source code analysis tools. But none of these tools work in cgo (that is, there is no way to troubleshoot).

Conversely, a good tool like valgrind doesn't understand Go's calling conventions or stack layout. At this point, Ian Lance Taylor's job is to integrate clang's memory sanitizer to debug C-side dangling pointers, which will benefit cgo users in Go 1.6.

The result of combining Go code and C code is the intersection of the two worlds, not the junction; the memory safety of C and the debuggability of Go programs. But lost space for many core tools.

Performance will always be an issue

C code and Go code live in two different worlds, cgo crosses the border between them, and this conversion is not free. And depending on where it lives in your code, the cost can be insignificant or huge.

C knows nothing about Go's calling conventions or growable stacks, so a call to C code must record all the details of the goroutine stack, switch to the C stack, and run the C code that has no idea how it was called , or the larger Go runtime responsible for the program knows nothing.

To be fair, Go doesn't know anything about the C world either. That's why the rules for passing data between the two are getting more and more cumbersome over time, as compilers are getting better at discovering stack data that is no longer considered valid, and garbage collectors are getting better Good at doing the same with heaps.

If there is a failure in the C world, the Go code must restore enough state, at least to print out the stack trace and exit the program cleanly, without exposing the core file information.

Managing this transition across the call stack, especially when it comes to signals, threads and callbacks, is not easy (Ian Lance Taylor also did a lot of work in Go 1.6 to improve the interoperability of signal handling with C ).

In the final analysis, the conversion between C language and Go language is not easy, both are ignorant of each other, and there will be obvious performance overhead .

The C language calls the shots, not your code

It doesn't matter which language you write bindings or wrapping C code in; Python, Java with JNI, some languages with libFFI, or Go via cgo; it's the world of C, and you just live in it.

Go code and C code have to agree on how to share resources like address space, signal handlers, and thread TLS slots -- and when I say agreement, I really mean that Go has to work around the assumptions of the C code. C code can assume that it is always running on one thread, or is simply not ready to work in a multithreaded environment.

You're not writing a Go program that uses the logic of a C library, you're writing a Go program that must coexist with mutually uncontrollable C code that is hard to replace, has the upper hand in negotiations, and doesn't care about your question.

Deployment becomes more complex

Any Go talk to a general audience will contain at least one slide with these words: Single, static binary.

This is a trump card for Go, making it a poster child for moving away from virtual machines and runtime management. With cgo, you're giving up on that, and you're giving up areas of Go's strengths.

Depending on your environment, you might compile your Go project into a deb or rpm, and assuming your other dependencies are also packaged, include them as install dependencies, pushing the problem to the OS's package manager . But this is a couple of major changes from the previous straightforward build and deployment process like go build && scp .

Compiling Go programs completely statically is possible, but by no means simple, suggesting that the effects of adding cgo to a project can ripple through the build and deployment lifecycle.

Wise choice

To put it bluntly, I'm not saying you shouldn't use cgo. But before you do this design, think carefully about the many qualities of Go that you will be giving up.

You need to think clearly about the pros and cons, and then consider whether it is worth it for you to do so.

The article is updated continuously, you can read it by searching on WeChat [Brain fried fish]. This article has been included in GitHub github.com/eddycjy/blog . If you are learning Go language, you can see the Go learning map and route . Welcome to Star to urge you to update.

Go Book Series

• Introduction to the Go language series: a preliminary exploration of the actual combat of the Go project
• Go Programming Journey: Deep Dive into Go Projects
• Go Language Design Philosophy: Understanding Go Why and Design Thinking
• Go Language Advanced Tour: Go deeper into the Go source code

read more

• The goto statement turns Go code into spaghetti?
• Go only if err != nil? It's not right to share these elegant handling poses with you!