[Go language] Detailed explanation of the context package that Xiaobai can understand: from entry to proficiency

Original link: Detailed explanation of the context package that Xiaobai can understand: from entry to proficiency

Preface

Hello, everyone, my name is asong . Today I want to share with you the context package. After a year of precipitation 1.17.1 , but the difference this time is that I plan to start from the beginning, because most beginner readers want to know first How to use it, and then you will care about how the source code is implemented.

I believe you will see such code in your daily work and development:

func a1(ctx context ...){
  b1(ctx)
}
func b1(ctx context ...){
  c1(ctx)
}
func c1(ctx context ...)

context is used as the first parameter (official recommendation), and it is continuously passed on. Basically, a project code is full of context , but do you really know what it does and how it works? I remember that when I first contacted context , my colleagues said that this is used for concurrency control. You can set a timeout, and the timeout will cancel the execution and return quickly. I simply think that as long as the function carries the context parameter to You can cancel the timeout and return quickly after the next pass. I believe that most beginners have the same idea with me. In fact, this is a wrong idea. The cancellation mechanism is also a notification mechanism. Pure transmission will not work. For example, if you write code like this:

func main()  {
    ctx,cancel := context.WithTimeout(context.Background(),10 * time.Second)
    defer cancel()
    go Monitor(ctx)

    time.Sleep(20 * time.Second)
}

func Monitor(ctx context.Context)  {
    for {
        fmt.Print("monitor")
    }
}

context passed through, it will not have any effect if it is not used. Therefore context . This article will start with the use and gradually analyze the context Go language. Let’s start now! ! !

The origin and function of the context

Looking at the official blog, we can know that the context package was introduced into the standard library in the go1.7

<img src="https://p3-juejin.byteimg.com/tos-cn-i-k3u1fbpfcp/c0d332914b0c44ae8706589eaef6ebaa~tplv-k3u1fbpfcp-zoom-1.image" style="zoom:50%;" />

context can be used to transfer context information between goroutine context can be passed to functions running in different goroutine . The context goroutine at the same time. The context package defines the context type and can be created background , TODO For a context, to propagate context between function call chains, you can also use WithDeadline , WithTimeout , WithCancel or WithValue replace it. It sounds a bit context is different from goroutine Inter-synchronization requests specific data, cancellation signals, and deadlines for processing the request.

At present, some of our commonly used libraries support context . For example, libraries gin database/sql context . This makes it more convenient for us to control concurrency. Just create a context context at the server entrance and continue to pass it through.

context

Created context

context package mainly provides two ways to create context :

• context.Backgroud()
• context.TODO()

These two functions are actually aliases for each other, there is no difference, the official definition is:

• context.Background is the default value of the context, and all other contexts should be derived from it.
• context.TODO should only be used when not sure which context should be used;

So in most cases, we use context.Background as the starting context to pass down.

The above two methods are to create the root context , which does not have any functions. The specific practice still depends on the With series of functions context

func WithCancel(parent Context) (ctx Context, cancel CancelFunc)
func WithDeadline(parent Context, deadline time.Time) (Context, CancelFunc)
func WithTimeout(parent Context, timeout time.Duration) (Context, CancelFunc)
func WithValue(parent Context, key, val interface{}) Context

These four functions must be Context . Through these functions, a Context tree is created. Each node of the tree can have any number of child nodes, and there can be any number of node levels. Draw a picture to show:

Based on a parent Context can be derived at will. In fact, this is a Context tree. Each node of the tree can have any number of child nodes, and there can be any number of node levels, and each child node depends on its parent node. For example, We can derive four children context Context.Background : ctx1.0-cancel , ctx2.0-deadline , ctx3.0-timeout , ctx4.0-withvalue . These four children context can also be used as the parent context other three nodes of 0618500b1c8505 will continue to be derived, even if the ctx1.0-cancel

That's all for creating the context method and the context derivative methods. Let's take a look at how they are used one by one.

WithValue carries data

In our daily business development, we hope to have a trace_id that can connect all the logs in series. This requires us to get this trace_id when we print the logs. In python we can use gevent.local to transmit, and in java we can use ThreadLocal to transmit , In the Go language, we can use Context to pass, by using WithValue to create a trace_id context , and then continue to pass on, and print the log output, just look at the usage example:

const (
    KEY = "trace_id"
)

func NewRequestID() string {
    return strings.Replace(uuid.New().String(), "-", "", -1)
}

func NewContextWithTraceID() context.Context {
    ctx := context.WithValue(context.Background(), KEY,NewRequestID())
    return ctx
}

func PrintLog(ctx context.Context, message string)  {
    fmt.Printf("%s|info|trace_id=%s|%s",time.Now().Format("2006-01-02 15:04:05") , GetContextValue(ctx, KEY), message)
}

func GetContextValue(ctx context.Context,k string)  string{
    v, ok := ctx.Value(k).(string)
    if !ok{
        return ""
    }
    return v
}

func ProcessEnter(ctx context.Context) {
    PrintLog(ctx, "Golang梦工厂")
}


func main()  {
    ProcessEnter(NewContextWithTraceID())
}

Output result:

2021-10-31 15:13:25|info|trace_id=7572e295351e478e91b1ba0fc37886c0|Golang梦工厂
Process finished with the exit code 0

We create a trace_id with ctx based on context.Background , and then pass it through the context tree. Any context derived from it will get this value. When we finally print the log, we can take the value from ctx and output it to the log. At present, some RPC frameworks support Context , so the downward transmission of trace_id

withVaule are four things to pay attention to when using 0618500b1c8788:

• Not recommended context value passed key parameters, declare the key parameters to be displayed out and should not be treated implicitly, context is best to carry the signature, trace_id such values.
• Because value is also key and value , in order to avoid context context multiple packages at the same time for 0618500b1c885f, key recommended to use the built-in type.
• In the above example, we get trace_id directly from the current ctx . In fact, we can also get the context in the value . When obtaining the key-value pair, we first context . If we don’t find it, we will look it up context The value corresponding to the key until context is returned nil or the corresponding value is found.
• context the data passed by key , value are all of interface . This type cannot be determined during compile time, so it is not very safe, so don't forget to ensure the robustness of the program when type assertion.

Timeout control

Usually robust programs need to set a timeout to avoid resource consumption due to long-time server response. Therefore, some web frameworks or rpc frameworks will use withTimeout or withDeadline for timeout control. When a request reaches the timeout time we set, it will It will be cancelled in time and will not be executed further. withTimeout and withDeadline have the same function, that is, the time parameters passed are different. They will automatically cancel Context through the passed time. It should be noted that they will all return a cancelFunc method. By calling this method, you can cancel in advance. However, it is recommended to call cancelFunc after the automatic cancellation to stop the timing to reduce unnecessary waste of resources.

withTimeout , WithDeadline except that WithTimeout duration time as input parameters rather than objects, which use these two methods are the same, see business scenarios and personal habits, because of the nature withTimout inside also called WithDeadline .

Now we give an example to try out the timeout control. Now we will simulate a request and write two examples:

• The next execution is terminated when the timeout period is reached

func main()  {
    HttpHandler()
}

func NewContextWithTimeout() (context.Context,context.CancelFunc) {
    return context.WithTimeout(context.Background(), 3 * time.Second)
}

func HttpHandler()  {
    ctx, cancel := NewContextWithTimeout()
    defer cancel()
    deal(ctx)
}

func deal(ctx context.Context)  {
    for i:=0; i< 10; i++ {
        time.Sleep(1*time.Second)
        select {
        case <- ctx.Done():
            fmt.Println(ctx.Err())
            return
        default:
            fmt.Printf("deal time is %d\n", i)
        }
    }
}

Output result:

deal time is 0
deal time is 1
context deadline exceeded

• The timeout period is not reached to terminate the next execution

func main()  {
    HttpHandler1()
}

func NewContextWithTimeout1() (context.Context,context.CancelFunc) {
    return context.WithTimeout(context.Background(), 3 * time.Second)
}

func HttpHandler1()  {
    ctx, cancel := NewContextWithTimeout1()
    defer cancel()
    deal1(ctx, cancel)
}

func deal1(ctx context.Context, cancel context.CancelFunc)  {
    for i:=0; i< 10; i++ {
        time.Sleep(1*time.Second)
        select {
        case <- ctx.Done():
            fmt.Println(ctx.Err())
            return
        default:
            fmt.Printf("deal time is %d\n", i)
            cancel()
        }
    }
}

Output result:

deal time is 0
context canceled

It is relatively easy to use, it can be automatically cancelled over time, and can also be manually controlled to cancel. context in the call link that is transparently transmitted from the request entry. If we want to open a goroutine in it to handle other things and it will not end with the request If it is cancelled later, then the transmitted context should be context.Background or context.TODO , and the veto will be inconsistent with expectations. You can read my previous article on the pit: A bug caused by improper context usage .

withCancel cancel control

In daily business development, we often open multiple gouroutine do some things in order to complete a complex requirement, which leads us to open multiple goroutine in one request, but we cannot control them, then we can use withCancel to derive A context passed to a different goroutine . When I want to goroutine , I can call cancel to cancel it.

Let's look at an example:

func main()  {
    ctx,cancel := context.WithCancel(context.Background())
    go Speak(ctx)
    time.Sleep(10*time.Second)
    cancel()
    time.Sleep(1*time.Second)
}

func Speak(ctx context.Context)  {
    for range time.Tick(time.Second){
        select {
        case <- ctx.Done():
            fmt.Println("我要闭嘴了")
            return
        default:
            fmt.Println("balabalabalabala")
        }
    }
}

operation result:

balabalabalabala
....省略
balabalabalabala
我要闭嘴了

We use withCancel create a Background , and then start a speech program to speak every 1s, main function executes cancel after 10s, then speak will exit when it detects the cancellation signal.

Custom Context

Because Context is essentially an interface, so we can achieve Context achieve custom Context purposes, typically implemented Web frame or RPC frame is frequently used to form such gin frame Context that they have a layer of packaging, and the specific code The implementation is posted here. If you are interested, you can see how gin.Context is implemented.

Appreciation of source code

Context is actually an interface, which defines four methods:

type Context interface {
 Deadline() (deadline time.Time, ok bool)
 Done() <-chan struct{}
 Err() error
 Value(key interface{}) interface{}
}

• Deadlne method: when Context automatically cancelled or it is cancelled when the cancellation time is reached, return
• Done method: when Context is cancelled or deadline return a closed channel
• Err method: when Context is cancelled or closed, return the reason for the cancellation of context
• Value method: get the value corresponding to the key

This interface is mainly inherited and implemented by three classes, namely emptyCtx , ValueCtx , cancelCtx , using the wording of an anonymous interface, so that any type that implements the interface can be rewritten.

Below we will analyze layer by layer from creation to use.

Create root Context

The object created when we call context.Background and context.TODO empty :

var (
    background = new(emptyCtx)
    todo       = new(emptyCtx)
)

func Background() Context {
    return background
}

func TODO() Context {
    return todo
}

Background and TODO still exactly the same, the official said: background is usually used by the main function, initialization and testing, and as the top-level context of the incoming request; TODO is when it is not clear which Context to use or is not yet available, the code should use the context. TODO is being replaced in the follow-up. In the final analysis, the semantics are different.

emptyCtx category

emptyCtx mainly used when we create the root Context . The implementation method is also an empty structure. The actual source code looks like this:

type emptyCtx int

func (*emptyCtx) Deadline() (deadline time.Time, ok bool) {
    return
}

func (*emptyCtx) Done() <-chan struct{} {
    return nil
}

func (*emptyCtx) Err() error {
    return nil
}

func (*emptyCtx) Value(key interface{}) interface{} {
    return nil
}

func (e *emptyCtx) String() string {
    switch e {
    case background:
        return "context.Background"
    case todo:
        return "context.TODO"
    }
    return "unknown empty Context"
}

WithValue

withValue internal part of 0618500b1ca83e is mainly to call the valueCtx class:

func WithValue(parent Context, key, val interface{}) Context {
    if parent == nil {
        panic("cannot create context from nil parent")
    }
    if key == nil {
        panic("nil key")
    }
    if !reflectlite.TypeOf(key).Comparable() {
        panic("key is not comparable")
    }
    return &valueCtx{parent, key, val}
}

valueCtx category

valueCtx purpose of Context is to carry key-value pairs for 0618500b1ca8ad, because it uses the inheritance implementation of anonymous interfaces, it will inherit the parent Context , which is equivalent to embedding in Context

type valueCtx struct {
    Context
    key, val interface{}
}

Implemented the String method to output Context and the key-value pair information carried:

func (c *valueCtx) String() string {
    return contextName(c.Context) + ".WithValue(type " +
        reflectlite.TypeOf(c.key).String() +
        ", val " + stringify(c.val) + ")"
}

Implement the Value method to store key-value pairs:

func (c *valueCtx) Value(key interface{}) interface{} {
    if c.key == key {
        return c.val
    }
    return c.Context.Value(key)
}

Look at the picture to understand:

So we call Context in Value will call up the layers until the final method when the root of the middle and if found key will return, otherwise will will find the ultimate emptyCtx return nil .

WithCancel

Let's take a look at the source code of the entry function of WithCancel

func WithCancel(parent Context) (ctx Context, cancel CancelFunc) {
    if parent == nil {
        panic("cannot create context from nil parent")
    }
    c := newCancelCtx(parent)
    propagateCancel(parent, &c)
    return &c, func() { c.cancel(true, Canceled) }
}

The execution steps of this function are as follows:

• Create a cancelCtx object as the child context
• Then call propagateCancel build the relationship between the father and son context , so that when the parent context is cancelled, the child context will also be cancelled.
• Return context object and subtree cancellation function

Let's analyze the class cancelCtx

cancelCtx Class 6

cancelCtx inherits Context and also implements the interface canceler :

type cancelCtx struct {
    Context

    mu       sync.Mutex            // protects following fields
    done     atomic.Value          // of chan struct{}, created lazily, closed by first cancel call
    children map[canceler]struct{} // set to nil by the first cancel call
    err      error                 // set to non-nil by the first cancel call
}

Short explanation:

• mu : It is a mutual exclusion lock to ensure concurrency safety, so context is concurrency safety
• done : used to make the cancellation notification signal of context chan struct{} type, now atomic.Value is used for lock optimization
• children : key is the interface type canceler , the purpose is to store canceler interface, when the root node is cancelled, traverse the child nodes to send a cancellation signal
• error : Store cancellation information context

Done method is implemented here, and the return is a read-only channel , the purpose is that we can wait for the notification signal channel

The specific code will not be posted. Let's go back and see propagateCancel is used to construct the Context between father and son 0618500b1cac45.

propagateCancel method

The code is a bit long and the explanation is a bit cumbersome. I added comments to the code and it seems more intuitive:

func propagateCancel(parent Context, child canceler) {
  // 如果返回nil，说明当前父`context`从来不会被取消，是一个空节点，直接返回即可。
    done := parent.Done()
    if done == nil {
        return // parent is never canceled
    }

  // 提前判断一个父context是否被取消，如果取消了也不需要构建关联了，
  // 把当前子节点取消掉并返回
    select {
    case <-done:
        // parent is already canceled
        child.cancel(false, parent.Err())
        return
    default:
    }

  // 这里目的就是找到可以“挂”、“取消”的context
    if p, ok := parentCancelCtx(parent); ok {
        p.mu.Lock()
    // 找到了可以“挂”、“取消”的context，但是已经被取消了，那么这个子节点也不需要
    // 继续挂靠了，取消即可
        if p.err != nil {
            child.cancel(false, p.err)
        } else {
      // 将当前节点挂到父节点的childrn map中，外面调用cancel时可以层层取消
            if p.children == nil {
        // 这里因为childer节点也会变成父节点，所以需要初始化map结构
                p.children = make(map[canceler]struct{})
            }
            p.children[child] = struct{}{}
        }
        p.mu.Unlock()
    } else {
    // 没有找到可“挂”，“取消”的父节点挂载，那么就开一个goroutine
        atomic.AddInt32(&goroutines, +1)
        go func() {
            select {
            case <-parent.Done():
                child.cancel(false, parent.Err())
            case <-child.Done():
            }
        }()
    }
}

What really puzzles this code is this if, else branch. Don't look at the code, just say why. Because we can customize context , context into a structure, it will not find the cancelable parent node, and we can only restart a coroutine to monitor.

If you are confused about this piece, I recommend reading Rao article: 1618500b1cad26 [Depth Deciphering the Go Language Context ] ( https://www.cnblogs.com/qcrao-2018/p/11007503.html), it will definitely be able to arrange it for you Worry about confusion.

cancel method

Finally, let's take a look at how the returned cancel method is implemented. This method will close the Channel in the context and cancel the signal synchronously to all sub-contexts:

func (c *cancelCtx) cancel(removeFromParent bool, err error) {
  // 取消时传入的error信息不能为nil, context定义了默认error:var Canceled = errors.New("context canceled")
    if err == nil {
        panic("context: internal error: missing cancel error")
    }
  // 已经有错误信息了，说明当前节点已经被取消过了
    c.mu.Lock()
    if c.err != nil {
        c.mu.Unlock()
        return // already canceled
    }
  
    c.err = err
  // 用来关闭channel，通知其他协程
    d, _ := c.done.Load().(chan struct{})
    if d == nil {
        c.done.Store(closedchan)
    } else {
        close(d)
    }
  // 当前节点向下取消，遍历它的所有子节点，然后取消
    for child := range c.children {
        // NOTE: acquiring the child's lock while holding parent's lock.
        child.cancel(false, err)
    }
  // 节点置空
    c.children = nil
    c.mu.Unlock()
  // 把当前节点从父节点中移除，只有在外部父节点调用时才会传true
  // 其他都是传false，内部调用都会因为c.children = nil被剔除出去
    if removeFromParent {
        removeChild(c.Context, c)
    }
}

At this point, the WithCancel method is analyzed. Through the source code, we can know that the cancel method can be called repeatedly and is idempotent.

withDeadline and WithTimeout

WithTimeout look at the 0618500b1cae15 method. Inside it is the WithDeadline method called:

func WithTimeout(parent Context, timeout time.Duration) (Context, CancelFunc) {
    return WithDeadline(parent, time.Now().Add(timeout))
}

So let’s focus on withDeadline is implemented:

func WithDeadline(parent Context, d time.Time) (Context, CancelFunc) {
  // 不能为空`context`创建衍生context
    if parent == nil {
        panic("cannot create context from nil parent")
    }
  
  // 当父context的结束时间早于要设置的时间，则不需要再去单独处理子节点的定时器了
    if cur, ok := parent.Deadline(); ok && cur.Before(d) {
        // The current deadline is already sooner than the new one.
        return WithCancel(parent)
    }
  // 创建一个timerCtx对象
    c := &timerCtx{
        cancelCtx: newCancelCtx(parent),
        deadline:  d,
    }
  // 将当前节点挂到父节点上
    propagateCancel(parent, c)
  
  // 获取过期时间
    dur := time.Until(d)
  // 当前时间已经过期了则直接取消
    if dur <= 0 {
        c.cancel(true, DeadlineExceeded) // deadline has already passed
        return c, func() { c.cancel(false, Canceled) }
    }
    c.mu.Lock()
    defer c.mu.Unlock()
  // 如果没被取消，则直接添加一个定时器，定时去取消
    if c.err == nil {
        c.timer = time.AfterFunc(dur, func() {
            c.cancel(true, DeadlineExceeded)
        })
    }
    return c, func() { c.cancel(true, Canceled) }
}

withDeadline compared to withCancel method also more a timer to regularly call cancel method, this cancel method in timerCtx rewritten class, we first look at timerCtx class, he is based cancelCtx , and two more fields :

type timerCtx struct {
    cancelCtx
    timer *time.Timer // Under cancelCtx.mu.

    deadline time.Time
}

timerCtx implemented cancel method, also called internal cancelCtx of cancel method to cancel:

func (c *timerCtx) cancel(removeFromParent bool, err error) {
  // 调用cancelCtx的cancel方法取消掉子节点context
    c.cancelCtx.cancel(false, err)
  // 从父context移除放到了这里来做
    if removeFromParent {
        // Remove this timerCtx from its parent cancelCtx's children.
        removeChild(c.cancelCtx.Context, c)
    }
  // 停掉定时器，释放资源
    c.mu.Lock()
    if c.timer != nil {
        c.timer.Stop()
        c.timer = nil
    }
    c.mu.Unlock()
}

Finally, we have finished reading the source code part, what do you think now?

context and disadvantages of 0618500b1caf67

context package is designed for concurrency control. This package has advantages and disadvantages. I have summarized several advantages and disadvantages. Welcome to add in the comment area.

shortcoming

• Affects the beauty of the code. Now basically all the web and RPC frameworks implement context , which leads to a parameter of each function in our code is context , even if you don’t need to take this parameter to pass it through, personally feel a little bit ugly.
• context can carry values, but there are no restrictions. There are no restrictions on the type and size, that is, there are no restrictions. This can easily lead to abuse, and the robustness of the program is difficult to guarantee; there is another problem that context is not as good as explicitly passing values Comfortable, the readability has deteriorated.
• You can customize context , so the risk is uncontrollable and will lead to abuse.
• context The error return of cancellation and automatic cancellation is not friendly enough, you cannot customize the error, and it is difficult to troubleshoot when there are problems that are difficult to troubleshoot.
• Creating a derivative node is actually creating a linked list node, and its time complexity is O(n). If there are more nodes, the efficiency of dropping branches will become lower.

advantage

• Using context can better control the concurrency and better manage the abuse of goroutine
• context , so we can transmit any data, which can be said to be a double-edged sword
• Online said context package to solve the goroutine of cancelation question, do you think?

Reference article

https://pkg.go.dev/context@go1.7beta1#Background
https://studygolang.com/articles/21531
https://draveness.me/golang/docs/part3-runtime/ch06-concurrency/golang-context/
https://www.cnblogs.com/qcrao-2018/p/11007503.html
https://segmentfault.com/a/1190000039294140
https://www.flysnow.org/2017/05/12/go-in-action-go-context.html

Summarize

context is a bit ugly in use, it can solve many problems. In daily business development, it is inseparable from context , but do not use the wrong context . The cancellation also uses the channel notification, so there are still some requirements in the code. There is a monitoring code to monitor the cancellation signal, which is often overlooked by the majority of beginners.

The example in the article has been uploaded github : https://github.com/asong2020/Golang_Dream/tree/master/code_demo/context_example

, this is the end of this article. I am asong . See you in the next issue.

**Welcome to follow the public account: [Golang DreamWorks]