Definition and reading of Go project configuration files

foreword

When we write applications, we basically use configuration files, from various shell to nginx , etc., all have their own configuration files. Although this is not too difficult, the configuration items are generally relatively complicated, and the parsing and verification will be more troublesome. This article will tell you how we simplify the definition and parsing of configuration files.

Scenes

If we want to write a service of Restful API , the configuration items probably have the following contents:

• Host , listening IP , if not filled, the default is 0.0.0.0
• Port , the listening port, required, can only be a number, greater than or equal to 80, less than 65535
• LogMode , log mode, only file or console
• Verbose , to see whether to output detailed logs, optional, the default is false
• MaxConns , the maximum number of concurrent connections allowed, default 10000
• Timeout , timeout setting, default 3s
• CpuThreshold ， CPU使用率触发系统降载的阈值，默认900 ， 1000m表示100%

Before, we used json as the configuration file, but json had a problem and could not add comments, so we later switched to the yaml format.

Next, let's see how to easily define and parse such a configuration file with the help of go-zero

define configuration

First, we need to define the above configuration requirements into the Go structure, as follows:

 RestfulConf struct {
    Host         string        `json:",default=0.0.0.0"`
    Port         int           `json:",range=[80,65535)"`
    LogMode      string        `json:",options=[file,console]"`
    Verbose      bool          `json:",optional"`
    MaxConns     int           `json:",default=10000"`
    Timeout      time.Duration `json:",default=3s"`
    CpuThreshold int64         `json:",default=900,range=[0:1000]"`
}

As you can see, we have certain definitions and restrictions for each configuration item, some of which are defined as follows:

• default , if the configuration is not filled, use the default value, you can see that the 3s will be automatically parsed into time.Duration type
• optional , this item can not be configured, if not, use the type zero value
• range , limited number type, needs to be within the given range
• options , the value of the limit configuration can only be one of the given ones

Also, some properties can be stacked, such as:

• default and range can be used together to increase the range limit and provide a default value
• default and options can be used together to increase optional restrictions and provide default values

configuration file

Because we gave a lot of default values when defining the configuration, and also used optional to specify as optional, so the configuration items in our configuration file are relatively few, and the default values can be used. There is no need to write, as follows:

 # 因为很多都有默认值，所以只需要写需要指定值和没有默认值的
Port: 8080
LogMode: console
# 可以读取环境变量的值
MaxBytes: ${MAX_BYTES}

There is a note here, if the configuration items value are all numbers, and the configuration type you define is string , for example, some people often use 123456 for testing passwords, However, the password is generally defined as string , and the configuration should be written as follows (just an example, the password is generally not recommended to be written in the configuration file):

 Password: "123456"

The double quotes here can’t be less, if there is less, it will report errors such as type mismatch 561cab0c5e374d5034e170c356216ef5---, because yaml parser will parse the int 123456 int .

load configuration file

We have the configuration definition ( config.go ) and the configuration file ( config.yaml ), the next step is to load the configuration file. There are three ways to load the configuration file:

• It must be loaded successfully, otherwise the program exits. We generally use this. If the configuration is incorrect, the program cannot continue.

 // 有错误直接退出程序
var config RestfulConf
conf.MustLoad("config.yaml", &config)

go-zero comes with goctl The generated default code also uses MustLoad to load the configuration file

• Load the configuration and judge by yourself whether there is error

 // 自己判断并处理 error
var config RestfulConf
// 为了更简洁，这里的 LoadConfig 后续会改为 Load，LoadConfig 已被标记为 Deprecated
if err := conf.LoadConfig("config.yaml", &config); err != nil {
    log.Fatal(err)
}

• Load configuration and read environment variables

 // 自动读取环境变量
var config RestfulConf
conf.MustLoad(configFile, &config, conf.UseEnv())

Why do we need to explicitly specify conf.UseEnv() , because if it is read by default, you may need escape when you write specific characters in the configuration, so the environment is not read by default Variables, this design also welcomes your suggestions.

Implementation principle

We generally use the encoding/json or the corresponding yaml library directly when we implement similar yaml/json parsing, but for go-zero , we We need to have more precise control when unmarshal , which requires us to customize the analysis of yaml/json . The complete code is implemented in:

Configuration file code: https://github.com/zeromicro/go-zero/tree/master/core/conf

yaml/json Parsing code: https://github.com/zeromicro/go-zero/tree/master/core/mapping

It also fully demonstrates the usage of reflect , and how to ensure the correctness of the code through unit testing in complex scenarios.

Summarize

I have always recommended the idea of Fail Fast . We are also doing the same when loading the configuration file. Once there is an error, exit immediately, so that the operation and maintenance will find the problem in time when deploying the service, because the process cannot be started at all. .

The configuration items of all services of go-zero are loaded and automatically verified in this way, including the configuration of many tools I wrote are also based on this, I hope it can be helpful to you!

project address

https://github.com/zeromicro/go-zero

Welcome go-zero and star support us!

WeChat exchange group

Follow the official account of " Microservice Practice " and click on the exchange group to get the QR code of the community group.