Go language restarts, the past and present and basic use of Go Modules

Click on "Cloud Recommendation Big Coffee", get the official recommended boutique content, and learn technology without getting lost!

With the development of the Go language and changes in scenarios, problems such as compilation dependencies, internal self-signed certificates, and security audits caused by GOPATH have emerged one after another. With the gradual improvement of the official Go Modules, users have new choices. This article will lead you to learn and use Go Modules from 0.

A developer report from Tencent in 2020 shows that the Go language has become the second largest back-end development language within Tencent. There are a large number of Go developers doing business and platform development at Tencent every day, and a large number of teams and projects are also exposed Some problems, with the emergence of Go Modules, problems such as internal self-issued certificates and security audits have gradually been resolved.

The author is currently responsible for Tencent Cloud’s use of the Go programming language. In early 2021, I will be responsible for internal goproxy services and promote the use of Go Modules. These technologies support Tencent Cloud, WeChat, Tencent Video, Tencent Games, Tencent Music, and Tencent Conferences. And other star products, and reached close cooperation with the company's internal software source team, worker bee team, TRPC team and various CI teams. In this series of articles, the author will help you start learning and understanding Go Modules from the simpler to the deeper.

Pattern evolution of Golang development

Since the birth of Go, users have been using the GOPATH environment variable. With the rapid development and growth of the Go language, the compilation dependency problems caused by GOPATH have gradually appeared. Finally in 2019, when Golang ushered in its 10th anniversary, the Google Go team finally began to focus on this environmental variable that has accompanied Golang for ten years.

Use of GOPATH

There are currently two development modes in Go, GOPATH mode and Go modules mode.
Before Go modules, the GOPATH development model was used for dependency management in Go development. In GOPATH development mode, the Go command uses the GOPATH environment variable to achieve the following functions:

1. The go install command installs the binary library to $GOBIN, and its default path is $GOPATH/bin.
2. The go install command installs the compiled package to $GOPATH/pkg/, for example, install example.com/y/z to $GOPATH/pkg/example.com/y/za.
3. The go get command downloads the source code package to $GOPATH/src/, for example, download example.com/y/z to $GOPATH/src/example.

The development history of Go modules

The GOPATH mode development model will eventually be eliminated. Go officially added the concept of package version to the entire Go development ecosystem, and then introduced the Go modules development model. It is a long process to switch from the GOPATH mode development model to Go modules. It has gone through several Go release versions:

• Go 1.11 (August 2018) introduced GO111MODULE environment variable, its default value is auto. If the variable GO111MODULE=off is set, then the go command will always use the GOPATH mode development mode. If you set this variable GO111MODULE=on, the go command will always use Go modules development mode. If the variable GO111MODULE=auto is set (or not set), the go command line will determine which mode to use according to the current working directory. If the current directory is outside of $GOPATH/src and there is a go.mod file in the root directory, then The go command will enable the Go module mode, otherwise the GOPATH development mode will be used. This rule ensures that all original compilations are not affected when the auto value is used in $GOPATH/src, and you can also experience the latest Go module development mode in other directories.
• Go 1.13 (August 2019) adjusted the restrictions on $GOPATH/src in GO111MODULE=auto mode. If a code base is in $GOPATH/src and there is a go.mod file, the go command will enable module development mode . This allows users to continue to organize their checkout code in an import-based hierarchy, but use modules for the import of individual repositories.
• Go 1.16 (February 2021) will set GO111MODULE=on as the default value and enable the go module development mode by default, which means that the GOPATH development mode will be completely closed by default. If users need to use GOPATH development mode, they can specify the environment variable GO111MODULE=auto or GO111MODULE=off.
• Go 1.NN (???) will discard the GO111MODULE environment variable and GOPATH development model, and use the module development model by default.

The love between GOPATH and Go modules wants to kill

In response to several questions that everyone is concerned about, the author makes the following answers:

Q1: Will the GOPATH variable be removed?
A: No, the GOPATH variable will not be removed. Abandoning the GOPATH development model in the future does not mean deleting the GOPATH environment variable, it will continue to be retained, and the main functions are as follows:

• The go install command installs the binary to the $GOBIN directory, and its default location is $GOPATH/bin.
• The go get command caches the downloaded modules to the $GOMODCACHE directory, the default location is $GOPATH/pkg/mod.
• The go get command caches the downloaded checksum data to the $GOPATH/pkg/sumdb directory.

Q2: Can I continue to create a code base GOPATH/src/import/path
A: Yes, many developers organize their repositories with this file structure. You only need to add the go.mod file to the repositories created by yourself.

Q3: If I want to test and modify a dependent library I need, how do I do it?
A: If you rely on some unreleased changes when compiling your own project, you can use go.mod's replace to achieve your needs.
For example, if you have downloaded golang.org/x/website and golang.org/x/tools to the $GOPATH/src/ directory, then you can download it in $GOPATH/src/golang.org/x/website/ Add the following instructions to go.mod to complete the replacement:
replace golang.org/x/tools => $GOPATH/src/golang.org/x/tools
Of course, the replace command is not aware of GOPATH, and the code can also be downloaded to other directories.
Use Go Modules from 0

1. Create a new Go module
   First create a new directory /home/gopher/hello, then enter this directory, and then create a new file,

hello.go:
package hello

func Hello() string {
    return "Hello, world."
}

Then write a corresponding test file hello_test.go:

package hello

import "testing"

func TestHello(t *testing.T) {
    want := "Hello, world."
    if got := Hello(); got != want {
        t.Errorf("Hello() = %q, want %q", got, want)
    }
}

Now we have a package, but it is not yet a module, because the go.mod file has not been created yet. If you execute go test in the /home/gopher/hello directory, you can see:

$ go test
go: go.mod file not found in current directory or any parent directory; see 'go help modules'

You can see that the Go command line prompt did not find the go.mod file, you can refer to go help modules. In this case, you can use Go mod init to initialize it, and then execute Go test:

$ go mod init example.com/hello
go: creating new go.mod: module example.com/hello
go: to add module requirements and sums:

go mod tidy
$go mod tidygo 
$ go test
PASS
ok      example.com/hello   0.020s
$

In this case, the module test is complete.
Then execute the go mod init command to create a go.mod file:

$ cat go.mod
module example.com/hello

go 1.17

2. Add dependency to module
   The main highlight of Go modules lies in the use of code written by others when programming, that is, a very good experience when introducing a dependent library. First update hello.go and introduce rsc.io/quote to implement some new functions.

package hello

import "rsc.io/quote"

func Hello() string {
    return quote.Hello()
}

Then, test it again:

$ go test
hello.go:3:8: no required module provides package rsc.io/quote; to add it:

go get rsc.io/quote
$ go get rsc.io/quote
go: downloading rsc.io/quote v1.5.2
go: downloading rsc.io/sampler v1.3.0
go: downloading golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c
go: added golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c
go: added rsc.io/quote v1.5.2
go: added rsc.io/sampler v1.3.0
$ go test
PASS
ok      example.com/hello   1.401s

Starting from Go 1.7, Go modules began to use lazyloading loading mechanism, and the update of dependent libraries needs to be updated manually according to the prompts.
The Go command will parse and pull the specified dependency version according to the go.mod file. If the specified version is not found in go.mod, it will prompt the corresponding command to guide the user to add it, and then the Go command will analyze the latest stable version (Latest) and add it to the go.mod file. In this example, you can see that the first run of Go test requires the dependency rsc.io/quote, but it is not found in the go.mod file, so the user is guided to get the latest version, and the user uses go get rsc. io/quote obtained the latest version v1.5.2, and also downloaded two other dependencies required by rsc.io/quote: rsc.io/sampler and golang.org/x/text. Indirect dependent references will also be recorded in the go.mod file, marked with indirect comments.

$ cat go.mod
module example.com/hello

go 1.17

require (
    golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c // indirect
    rsc.io/quote v1.5.2 // indirect
    rsc.io/sampler v1.3.0 // indirect
)

Running the go test command again will not repeat the above work, because go.mod is already up-to-date, and the required dependencies have been downloaded to the machine (in $GOPATH/pkg/mod):

$ go test
PASS
ok      example.com/hello   0.020s

Note that although the Go command can add new dependencies quickly and easily, it is not without cost.
As mentioned above, adding a direct dependency to the project may introduce other indirect dependencies. The go list -m all command can list all the dependencies that the current project depends on:

$ go list -m all
example.com/hello
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c
rsc.io/quote v1.5.2
rsc.io/sampler v1.3.0

In the output of go list, you can see that the current module, also known as the main module, will be displayed on the first line, and then the others will be sorted according to the module path. Among them, the version number v0.0.0-20170915032832-14c0d48ead0c that relies on golang.org/x/text is a pseudo version number, which is a kind of Go version and points to a commit that is not tagged.
In addition to the go.mod file, the go command also maintains a file called go.sum, which contains the cryptographic hash value corresponding to each version.

$ cat go.sum
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c h1:qgOY6WgZO...
golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c/go.mod h1:Nq...
rsc.io/quote v1.5.2 h1:w5fcysjrx7yqtD/aO+QwRjYZOKnaM9Uh2b40tElTs3...
rsc.io/quote v1.5.2/go.mod h1:LzX7hefJvL54yjefDEDHNONDjII0t9xZLPX...
rsc.io/sampler v1.3.0 h1:7uVkIFmeBqHfdjD+gZwtXXI+RODJ2Wc4O7MPEh/Q...
rsc.io/sampler v1.3.0/go.mod h1:T1hPZKmBbMNahiBKFy5HrXp6adAjACjK9...

The Go command uses go.sum to ensure that the dependency library code downloaded each time is consistent with the first time, so as to ensure that there will be no abnormalities in the project, so go.mod and go.sum should be uploaded to git and other versions In the control system.

3. Update dependencies
   From the output of the go list -m all command above, you can see that a pseudo version number is used in the library golang.org/x/text. First update this version to the latest stable version:

$ go get golang.org/x/text
go: downloading golang.org/x/text v0.3.7
go: upgraded golang.org/x/text v0.0.0-20170915032832-14c0d48ead0c => v0.3.7

$ go test
PASS
ok      example.com/hello   0.013s

The test can still pass, and then execute go list -m all:

$ go list -m all
example.com/hello
golang.org/x/text v0.3.7
rsc.io/quote v1.5.2
rsc.io/sampler v1.3.0

$ cat go.mod
module example.com/hello

go 1.17

require (
    golang.org/x/text v0.3.7 // indirect
    rsc.io/quote v1.5.2 // indirect
    rsc.io/sampler v1.3.0 // indirect
)

Concluding remarks

Go unifies the dependency management of many third parties in the Go ecosystem through the dependency management of Go modules, and is highly integrated in the Go command line, without additional installation and use by developers. Go is currently supported in the currently maintained Go version. modules. For users who have not switched to Go modules, it is strongly recommended that everyone start using it. This provides many high-quality features and guarantees in terms of team development experience, performance, and security.

"Yunjian Big Coffee" is a special column for Tencent's cloud plus community. Cloud recommendation officials specially invite industry leaders to focus on the implementation of cutting-edge technologies and theoretical practice, and continue to interpret hot technologies in the cloud era and explore new opportunities for industry development. Click one-click to subscribe to , and we will regularly push premium content for you.