Introduction to This problem stems from the fact that we want to make a change to the module path of dubbo-go, using dubbo.apache.org/dubbo-go/v3 to replace the previous github.com/apache/dubbo-go.
Author | Dong Jianhui, Sheng Aofei
Source | Alibaba Cloud Native Public
Problem background
The problem stems from the fact that we wanted to make a change to the module path of dubbo-go, replacing the previous dubbo.apache.org/dubbo-go/v3 github.com/apache/dubbo-go .
First, we did a path mapping and placed a dubbogo/v3 file under dubbo.apache.org, the content is as follows:
<html>
<head>
<meta name="go-import" content="dubbo.apache.org/dubbo-go/v3 git <https://github.com/apache/dubbo-go>">
<meta name="go-source" content="dubbo.apache.org/dubbo-go/v3 git <https://github.com/apache/dubbo-go/tree/3.0{/dir}> <https://github.com/apache/dubbo-go/blob/3.0{/dir}/{file}#L{line}>">
<meta http-equiv="refresh" content="0; url=https://pkg.go.dev/dubbo.apache.org/dubbo-go/v3">
</head>
<body>
<p>Redirecting to <a href="<https://pkg.go.dev/dubbo.apache.org/dubbo-go/v3>">pkg.go.dev/dubbo.apache.org/dubbo-go/v3</a>...</p>
</body>
</html>
Secondly, we modified the module of go.mod and all the corresponding imports, and modified the module path used by all submodules to reference dubbo-go.
problem analysis
After making the above modifications, we found that CI failed when we raised the PR. After further log investigation, we determined that CI had an error during the integration test. The specific error message is as follows:
The execution logic of this section is to use docker to build an image of the integrated test content in dubbo-go and start the container for testing. The Dockerfile path used for the image packaging is in the github.com/apache/dubbo-go/test/integrate/dubbo/go-server directory. Check the STEP ID of the error log, we can locate it The specific error occurs in the following two steps:
# ...
# STEP 9
RUN test ${PR_ORIGIN_REPO} && go mod edit -replace=dubbo.apache.org/dubbo-go/v3=github.com/{PR_ORIGIN_REPO}@${PR_ORIGIN_COMMITID} || go get -u dubbo.apache.org/dubbo-go/v3@develop
# ...
# STEP 11
RUN go mod tidy && go install github.com/apache/dubbo-go/test/integrate/dubbo/go-server
In STEP 9, we go mod edit -replace with 060c6db51ef29a, and replaced it with the warehouse address and commit id that initiated the PR request. On this basis, when the image build ran to STEP11, an error occurred when go mod tidy
Looking back at the error log, we can find:
Since we only specified the commit id, in the actual running process of go mod, it will generate a hypothetical version number for us (for more description of the hypothetical version number, question extension end of this article), this hypothetical version number The latest valid tag for grabbing the remote warehouse is v1.4.1 [Note: The remote warehouse here is github.com/Mulavar/dubbo-go, this is my own dubbo-go branch, and the fork was pulled earlier by this branch. The last tag is v1.4.1], and it is increased to v1.4.2, and the main version is v1. But before, our dubbogo module path was declared as dubbo.apache.org/dubbogo/v3 and the main version was v3. This resulted in go mod edit -replace being v3 and v1 respectively. There was an inconsistency, and an error occurred when actually pulling the dubbogo dependencies.
problem solved
In the problem analysis section, we locate this problem in STEP 9 of mirroring construction, namely:
# STEP 9
RUN test ${PR_ORIGIN_REPO} && go mod edit -replace=dubbo.apache.org/dubbo-go/v3=github.com/{PR_ORIGIN_REPO}@${PR_ORIGIN_COMMITID} || go get -u dubbo.apache.org/dubbo-go/v3@develop
In this step, when we use go mod edit -replace , a v3 version of the go module with a v1 version of the module, resulting in inconsistent major versions and error , so we only need to specify the major version of the replaced module when replacing the dependency path. V3 is o mod edit -replace . We also add the v3 suffix to the module dependency path after g 060c6db51ef337 as shown below.
before fixing:
go mod edit -replace=dubbo.apache.org/dubbo-go/v3=github.com/{PR_ORIGIN_REPO}@${PR_ORIGIN_COMMITID}
⇒ After modification:
go mod edit -replace=dubbo.apache.org/dubbo-go/v3=github.com/${PR_ORIGIN_REPO}/v3@${PR_ORIGIN_COMMITID}
After fixing the problem, we submit the code to view CI. In the log information printed by STEP 8, you can see that the replaced dubbogo path has more than v3, and the version number ( v3.0.0-20210509140455-2574eab5ad0b ) followed when the package is pulled is also v3. Successfully pulled the dependency.
Problem development
1. Semantic Import Versioning
When we use Go modules to build the dependencies of the project, the dependencies of the go project need to declare the version number we use. Considering that compatibility is always the most important and headache for developers every time a new version is released, so go through Semantic Import Versioning to formulate the version number standard to ensure that each developer can specify the dependent version to use according to their own project requirements. According to go's semantic import version control guidelines, the version number consists of three parts:
Note: The above picture and part of the content above refer to the article "Go Modules Detailed".
Among them, Major version indicates that this is a new major version, and even this new version may be incompatible with the old version.
The Minor version indicates that this is an iteration in a major version, which is mainly used for incrementing when new features are added.
Patch version is the most granular version update, indicating some bug fixes.
There are two ways to specify the major version. One is to declare directly as above, and the other is to add a version suffix at the end of the project path. For example, the module declared in version 3.0 of dubbo.apache.org/dubbo-go/v3 is 060c6db51ef44e, where v3 is a major version. statement of.
2. pseudo-version
Go recommends that we specify a clear version number when using dependencies, such as the dependency on 060c6db51ef47d declared in cloud.google.com/go 's go.mod:
But considering that in some scenarios, we want to use an unreleased version of the module dependency (the module has only a known commit id), such as the github.com/Microsoft/go-winio dependency declared by dubbogo, we can use an assumed version number ( pseudo-version ) to replace the real version number. Assume that the format of the version number is:
// (1) vX.0.0-yyyymmddhhmmss-abcdef123456
// (2) vX.Y.(Z+1)-0.yyyymmddhhmmss-abcdef123456
// (3) vX.Y.(Z+1)-0.yyyymmddhhmmss-abcdef123456+incompatible
// (4) vX.Y.Z-pre.0.yyyymmddhhmmss-abcdef123456
// (5) vX.Y.Z-pre.0.yyyymmddhhmmss-abcdef123456+incompatible
It can be seen that pseudo-version is divided into three parts by short slashes. The first part XYZ is the same as mentioned in Section 4.1, which are major version , minor version , patch version , and the second part is a timestamp in the format yyyymmddhhmmss, and the third part is A 12-digit commit hash can be specified manually, if not, it will be automatically generated by go.
Regarding pseudo-version , the generation rules of go are as follows:
- Query the latest tag of the major version corresponding to the project (if the project dependency path does not have an explicit v2/v3 suffix, the default is v1 version).
- If there is a tag, use the latest tag to increment the patch version.
- Without tag, it is automatically generated according to the suffix version number of the project path (for example, v3 will automatically generate a
pseudo-versionstarting with 3.0.0).
Following this rule, looking back at the problem analysis and problem solving in the previous article, we can understand why dubbo-go's pseudo-version was v1.4.2 at the beginning. This is precisely because go believes that the dubbogo after replacement depends on the main version of v1, so we can find it. The latest tag under the major version is added, and patch version incremented, which leads to inconsistencies between the previous and previous major versions. When we add v3 to the version path, go cannot find the tag under the corresponding major version, and automatically generates v3.0.0 for us. , Which passed CI.
3. Go module nesting
Unlike Java, go has no concept of submodules. Even sometimes, we will see multiple Go modules in a repo, such as a go.mod in the root directory of the project, and go.mod in some subdirectories.
This is called a nested module in Go modules, not a parent-child module, that is, the have nothing to do with each other and are independent of each other .
When do you need a single repo and multiple modules? Generally speaking, we will consider using the single repo multi-module development form when we encounter the following two situations.
- A certain nested module changes very frequently and needs to be released frequently.
- The nested modules only depend on a fixed version of the root module.
In both cases, the essence is that there is no strong version binding relationship between the two modules, but due to some other reasons, they need to be placed under one rpeo, thus forming a single repo with multiple modules.
4. Dubbogo static mapping file content analysis
dubbo-go uses static file mapping to implement module redirection. The content of the static file is as follows:
The core part is the meta tags go-import and go-source .
<html>
<head>
<meta name="go-import" content="dubbo.apache.org/dubbo-go/v3 git <https://github.com/apache/dubbo-go>">
<meta name="go-source" content="dubbo.apache.org/dubbo-go/v3 git <https://github.com/apache/dubbo-go/tree/3.0{/dir}> <https://github.com/apache/dubbo-go/blob/3.0{/dir}/{file}#L{line}>">
<meta http-equiv="refresh" content="0; url=https://pkg.go.dev/dubbo.apache.org/dubbo-go/v3">
</head>
<body>
<p>Redirecting to <a href="<https://pkg.go.dev/dubbo.apache.org/dubbo-go/v3>">pkg.go.dev/dubbo.apache.org/dubbo-go/v3</a>...</p>
</body>
</html>
1）go-import
go-import role is to tell go get where to find the source, content is divided into three parts:
dubbo.apache.org/dubbo-go/v3: The module statement of this project.git: The version control tool used.<https://github.com/apache/dubbo-go>: Tell go get where to find the source code for this project.
2）go-source
go-source is to generate a specific go doc (now pkg.go.dev) document for the project. There are 4 parts in total. The first two parts go-import , which are the module declaration and version control tool of the project. The latter two parts are The functions are as follows:
<https://github.com/apache/dubbo-go/tree/3.0{/dir}>: Declare the location of the source code of the project.<https://github.com/apache/dubbo-go/blob/3.0{/dir}/{file}#L{line}>: Map documents and codes to help us jump to the corresponding specific content when we click on the directory tree of the document.
For example, on https://pkg.go.dev/dubbo.apache.org/dubbo-go/v3 , we click on one of the files:
Will jump to the document corresponding to the corresponding file.
Welcome students who are interested in the apache/dubbo-go project to search Dingding group number 31363295 and join the Dingding exchange group!
Reference
- 《The Go Blog — Using Go Modules》：https://blog.golang.org/using-go-modules
- 《Go Modules Reference》：https://golang.org/ref/mod
- "How to release Go Module v2 and above": https://blog.cyeam.com/go/2019/03/12/go-version
- "Go Modules Detailed": https://www.sulinehk.com/post/go-modules-details/
About the Author
Jianhui (github @Mulavar) , who has just graduated from the VLIS laboratory of Zhejiang University, and is currently working as a development engineer for the Yuanjiao service end.
Sheng Aofei (github @aofei) , the author of the goproxy.cn project, has submitted many PRs to the go source code (such as mod tools).
Copyright Statement: content of this article is contributed spontaneously by Alibaba Cloud real-name registered users, and the copyright belongs to the original author. The Alibaba Cloud Developer Community does not own its copyright and does not assume corresponding legal responsibilities. For specific rules, please refer to the "Alibaba Cloud Developer Community User Service Agreement" and the "Alibaba Cloud Developer Community Intellectual Property Protection Guidelines". If you find suspected plagiarism in this community, fill in the infringement complaint form to report it. Once verified, the community will immediately delete the suspected infringing content.
**粗体** _斜体_ [链接](http://example.com) `代码` - 列表 > 引用。你还可以使用@来通知其他用户。