Go daily development and standing third-party libraries and tools

Go has been almost a year since I wrote 0618329976cb9a unconsciously, and several projects, large and small, have been launched; my mentality has also undergone several rounds of changes.

Because I personally wrote Java about five years ago, and wrote 0618329976cbc2 halfway through for more than a Python , so the feeling when I first came into contact with Go is as follows:

There is Java the ecology of Python syntactic sugar of 0618329976cbf2.

The feeling I write up to now is:

I won't discuss who is strong and who is weak in these languages; I will focus on sharing some third-party libraries and tools used in our daily development.

Here I mainly divide these libraries into two categories:

• Business development
• Basic tool development

Business development

The first is business development, which mainly includes web , database, Redis etc.

Gin ⭐️⭐️⭐️⭐️⭐️

The first is Gin , an HTTP framework, simple to use, excellent performance, and a lot of information; when you are still hesitating which framework to choose, choose it, basically right.

Of course, the github.com/swaggo/gin-swagger swagger tool that matches it is just needed; use it to generate swagger documents.

GORM ⭐️⭐️⭐️⭐️⭐️

GORM has nothing to say, if you like orm to operate the database, then choose it; the same is also simple to use and more information.

If there is demand for separate read and write, you can use GORM plug-ins provided by the official https://github.com/go-gorm/dbresolver , with GORM use is also very simple.

errors ⭐️⭐️⭐️⭐️⭐️

The error handling provided by the Go language itself is relatively simple. https://github.com/pkg/errors provides more powerful functions, such as:

• Packaging abnormal
• Packing stack, etc.

The following APIs are commonly used:

// WithMessagef annotates err with the format specifier.
func WithMessagef(err error, format string, args ...interface{}) error

// WithStack annotates err with a stack trace at the point WithStack was called.
func WithStack(err error) error

zorolog ⭐️⭐️⭐️⭐️⭐️

There are many log printing libraries in Go, and it is best for logs to have a low sense of existence in daily development; that is to say, they have strong performance (cannot affect business code) and simple use of APIs.

"github.com/rs/zerolog/log"
log.Debug().Msgf("OrderID :%s", "12121")

excelize

https://github.com/qax-os/excelize is a library that reads and writes Excel, basically all Excel operations you can encounter can be implemented.

now ⭐️⭐️⭐️⭐️

https://github.com/jinzhu/now is a time tool library:

• Get the current year, month, day, hour, minute, and second.
• Support in different time zones.
• The last week, the last month, etc.

import "github.com/jinzhu/now"

time.Now() // 2013-11-18 17:51:49.123456789 Mon

now.BeginningOfMinute()        // 2013-11-18 17:51:00 Mon
now.BeginningOfHour()          // 2013-11-18 17:00:00 Mon
now.BeginningOfDay()           // 2013-11-18 00:00:00 Mon
now.BeginningOfWeek()          // 2013-11-17 00:00:00 Sun
now.BeginningOfMonth()         // 2013-11-01 00:00:00 Fri
now.BeginningOfQuarter()       // 2013-10-01 00:00:00 Tue
now.BeginningOfYear()          // 2013-01-01 00:00:00 Tue

now.EndOfMinute()              // 2013-11-18 17:51:59.999999999 Mon
now.EndOfHour()                // 2013-11-18 17:59:59.999999999 Mon
now.EndOfDay()                 // 2013-11-18 23:59:59.999999999 Mon
now.EndOfWeek()                // 2013-11-23 23:59:59.999999999 Sat
now.EndOfMonth()               // 2013-11-30 23:59:59.999999999 Sat
now.EndOfQuarter()             // 2013-12-31 23:59:59.999999999 Tue
now.EndOfYear()                // 2013-12-31 23:59:59.999999999 Tue

now.WeekStartDay = time.Monday // Set Monday as first day, default is Sunday
now.EndOfWeek()                // 2013-11-24 23:59:59.999999999 Sun

Decimal ⭐️⭐️⭐️⭐️

https://github.com/shopspring/decimal can help when accuracy calculations are needed in business.

import (
    "fmt"
    "github.com/shopspring/decimal"
)

func main() {
    price, err := decimal.NewFromString("136.02")

    quantity := decimal.NewFromInt(3)
    fee, _ := decimal.NewFromString(".035")
    taxRate, _ := decimal.NewFromString(".08875")

    subtotal := price.Mul(quantity)

    preTax := subtotal.Mul(fee.Add(decimal.NewFromFloat(1)))

    total := preTax.Mul(taxRate.Add(decimal.NewFromFloat(1)))

    fmt.Println("Subtotal:", subtotal)                      // Subtotal: 408.06
    fmt.Println("Pre-tax:", preTax)                         // Pre-tax: 422.3421
    fmt.Println("Taxes:", total.Sub(preTax))                // Taxes: 37.482861375
    fmt.Println("Total:", total)                            // Total: 459.824961375
    fmt.Println("Tax rate:", total.Sub(preTax).Div(preTax)) // Tax rate: 0.08875
}

Basically, it can do any precision conversion you can think of; with GORM you can also declare the model decimal , and the database corresponds to decimal , which will be more convenient to use.

Amount decimal.Decimal `gorm:"column:amout;default:0.0000;NOT NULL" json:"amout"` 

configor ⭐️⭐️⭐️⭐️

https://github.com/jinzhu/configor is a configuration file reading library that supports YAML/JSON/TOML such as 0618329976d184.

go-cache ⭐️⭐️⭐️

https://github.com/patrickmn/go-cache is a Java is similar Guava cache , thread-safe, easy to use; does not require a distributed caching simple scenario can be considered.

    c := cache.New(5*time.Minute, 10*time.Minute)
    // Set the value of the key "foo" to "bar", with the default expiration time
    c.Set("foo", "bar", cache.DefaultExpiration)

copier ⭐️⭐️⭐️

https://github.com/jinzhu/copier looking at the name, you know that this is a data replication library, similar to Java in BeanUtils.copy() struct with the same fields but different objects, and it also supports deep copy.

func Copy(toValue interface{}, fromValue interface{}) (err error) 

It is very useful when we need a temporary struct to store data, especially when there are a lot of fields in a struct, it is really a bit finger-consuming to assign values back and forth one by one.

But also be careful not to use it in all situations, it will bring some disadvantages:

• When deleting a field, you cannot use the compiler prompt.
• When some fields require additional manual processing, the code is not easy to read.
• Reflective assignment has a certain performance loss.

In short, in business development, it is recommended to write manually, after all, the code is for people to see.

env ⭐️⭐️⭐️

https://github.com/caarlos0/env This library can convert our environment variables into a struct .

type config struct {
    Home string `env:"HOME"`
}

func main() {
    cfg := config{}
    if err := env.Parse(&cfg); err != nil {
        fmt.Printf("%+v\n", err)
    }

    fmt.Printf("%+v\n", cfg)
}

This is very useful when we package the code into different runtime environments, and use it to easily obtain different environment variables.

user_agent ⭐️⭐️⭐️

https://github.com/mssola/user_agent is a small tool for user-agent

When we need to collect user-agen on the server side, we can read data faster.

func main() {
    ua := user_agent.New("Mozilla/5.0 (Linux; U; Android 2.3.7; en-us; Nexus One Build/FRF91) AppleWebKit/533.1 (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1")

    fmt.Printf("%v\n", ua.Mobile())   // => true
    fmt.Printf("%v\n", ua.Bot())      // => false
    fmt.Printf("%v\n", ua.Mozilla())  // => "5.0"
    fmt.Printf("%v\n", ua.Model())    // => "Nexus One"
    fmt.Printf("%v\n", ua.Platform()) // => "Linux"
    fmt.Printf("%v\n", ua.OS()) 
    }

phonenumbers ⭐️⭐️⭐️

https://github.com/nyaruka/phonenumbers mobile phone number verification library, you don’t need to write regular expressions yourself.

// parse our phone number
num, err := phonenumbers.Parse("6502530000", "US")

Basic tools

Next is some basic tool libraries, including some mainstream storage clients, middleware, etc.

gomonkey ⭐️⭐️⭐️⭐️⭐️

github.com/agiledragon/gomonkey is a mock piling tool. When we write unit tests, it mock on some non-interface functions, so we need to use it.

Because it modifies the machine jump instruction when calling the corresponding function, and the corresponding instruction is different for different CPU architectures, it is not compatible with Apple's M1 chip when we use it, but it should be compatible now, you can try it.

goconvey ⭐️⭐️⭐️⭐️⭐️

https://github.com/smartystreets/goconvey is also a library for unit testing, compatible with the go test command.

• Provide a visual web UI.
• Integrated with IDE to display unit coverage.
  

dig ⭐️⭐️⭐️⭐️⭐️

https://github.com/uber-go/dig This is a dependency injection library. We will not discuss whether we should use dependency injection for the time being. At least for now, we still have several advantages when using it:

• All objects are singletons.
• There is a unified place to manage objects.
• When in use, you can directly pass the object as a parameter (the container will automatically inject it).

Of course, there are also some inconvenient places:

• When unfamiliar, it is not clear how an object is created.
• The code is not very understandable.

We have developed a business framework internally, in which all objects are managed by dig, which is more convenient to use.

cobra ⭐️⭐️⭐️⭐️

https://github.com/spf13/cobra is a powerful command-line tool library, we use it to implement internal command-line tools, and also recommended https://github.com/urfave/cli / I personally would be more accustomed to using the latter, which should be more concise.

BloomRPC ⭐️⭐️⭐️⭐️

https://github.com/uw-labs/bloomrpc a gRPC visualization tool, which is much simpler gRPC

But there are some minor issues, such as accuracy. If int64 exceeds 2^56 the value obtained by the server will cause an error, which has not been resolved yet.

redis ⭐️⭐️⭐️⭐️

https://github.com/go-redis/redis/ Redis client, there is not much to say; after many years of development, all the functions are available.

elastic ⭐️⭐️⭐️⭐️

https://github.com/olivere/elastic This is also a very mature elasticsearch library.

resty ⭐️⭐️⭐️⭐️

https://github.com/go-resty/resty/ an http client, which is very simple to use:

// Create a Resty Client
client := resty.New()
resp, err := client.R().
    EnableTrace().
    Get("https://httpbin.org/get")

It smells like the Python requests

pulsar-client-go ⭐️⭐️⭐️

Pulsar's official go language client, compared to Java, clients in other languages are almost all stepmothers; there will be fewer functions, and updates are not so active; but there is no choice.

go-grpc-middleware ⭐️⭐️⭐️

https://github.com/grpc-ecosystem/go-grpc-middleware official of gRPC middleware, you can achieve some of their own internal authentication, metadata, log functions.

go-pilosa ⭐️⭐️⭐️

https://github.com/pilosa/go-pilosa is a bitmap database client. The scene application of bitmap database is limited, usually only when there is a label requirement; for example, if you want N labels Submit and supplement collection; after the data has a certain scale, operations will definitely raise relevant requirements; it can be prepared for emergencies.

pb ⭐️⭐️⭐️

https://github.com/cheggaaa/pb a command line tool progress bar, it will be more elegant to interact with it when writing command line tools.

Summarize

Finally, I have compiled a table for easy viewing:

name	type	Function	Star rating
Gin	Business development	HTTP frame	⭐️⭐️⭐️⭐️⭐️
GORM	Business development	ORM frame	⭐️⭐️⭐️⭐️⭐️
errors	Business development	Exception handling library	⭐️⭐️⭐️⭐️⭐️
zorolog	Business development	Log library	⭐️⭐️⭐️⭐️⭐️
excelize	Business development	Excel related requirements	⭐️⭐️⭐️⭐️⭐️
now	Business development	Time processing	⭐️⭐️⭐️⭐️️
Decimal	Business development	Precision processing	⭐️⭐️⭐️⭐️️
configor	Business development	Configuration file	⭐️⭐️⭐️⭐️️
go-cache	Business development	Local cache	⭐️⭐️⭐️
copier	Business development	Data replication	⭐️⭐️⭐️️️
env	Business development	Environment variable	⭐️⭐️⭐️️️
user_agent	Business development	Read user-agent	⭐️⭐️⭐️️️
phonenumbers	Business development	Mobile phone number verification	⭐️⭐️⭐️️️
gomonkey	Basic tools	mock tools	⭐️⭐️⭐️⭐️⭐
goconvey	Basic tools	Single test coverage	⭐️⭐️⭐️⭐️⭐
dig	Basic tools	Dependency injection	⭐️⭐️⭐️⭐️⭐
cobra	Basic tools	Command line tool	⭐️⭐️⭐️⭐
cli	Basic tools	Command line tool	⭐️⭐️⭐️⭐
BloomRPC	Basic tools	gRPC debug client	⭐️⭐️⭐️⭐
redis	Basic tools	Redis client	⭐️⭐️⭐️⭐
elastic	Basic tools	elasticsearch client	⭐️⭐️⭐️⭐
resty	Basic tools	http client	⭐️⭐️⭐️⭐
pulsar-client-go	Basic tools	Pulsar client	⭐️⭐️⭐️
go-grpc-middleware	Basic tools	gRPC middleware	⭐️⭐️⭐
go-pilosa	Basic tools	pilosa client	⭐️⭐️⭐️
pb	Basic tools	Command line tool progress bar	⭐️⭐️⭐️

The rule of star rating mainly depends on the frequency of actual use.

In the end, some private goods are entrained
The article mentioned that we have integrated a business development framework based on the above library; also based on this framework, we have launched more than 10 projects, large and small, and there is still a lot of room for improvement, and it is still in rapid iteration.

Approximate usage, entrance main.go :


Finally, I intercepted my internal sharing and summarized the overall thought -quoted from a colleague with the surname of the company.

Maybe after many iterations internally, we will try to open source when we feel that we are able to open up and bring some help to the community; at this stage, it is not too ugly.

These libraries are the most commonly used in our daily development, and you are welcome to leave your commonly used libraries and tools in the comment area.