Go compilation principle series 8 (variable capture)

foreword

In the previous articles, some work on the front end of the compiler has been basically shared. The following articles are mainly about the compiler's analysis and reconstruction of the abstract syntax tree, and then complete a series of optimizations, including the following five parts :

• variable capture
• function inlining
• escape analysis
• Closure rewrite
• Traversal function

The next five articles are mainly about the above five topics. This article shares variable capture. Variable capture is mainly for closure scenarios, because variables outside the closure may be referenced in the closure function, so variable capture needs to be clearly defined in the closure. Variables are captured in the package by value reference or address reference

Overview of variable capture

Let's take an example to see what variable capture is

 package main

import (
    "fmt"
)

func main() {
    a := 1
    b := 2
    go func() {
        //在闭包里对a或b进行了重新赋值，也会改变引用方式
        fmt.Println(a, b)
    }()
    a = 666
}

We can see that the external variables a and b are referenced in the closure. Since the variable a has undergone other assignment operations after the closure, the reference methods of the a and b variables in the closure will be different. In the closure, the variable a must be operated by address reference , and the reference to the variable b will be passed by direct value .

We can view the current program closure variable capture in the following ways

 go tool compile -m=2 main.go | grep capturing

assign=true means that the variable a is assigned again after the closure is completed

Also see a slightly more complicated

 func adder() func(int) int {//累加器
    sum := 0 //地址引用
    return func(v int) int {
        sum += v
        return sum
    }
}

func main() {
    a := adder()
    for i:=0;i<10;i++{
        fmt.Printf("0 + 1 + ... + %d = %d\n", i, a(i))
    }
}

The previous article shared type checking, we can continue to look down the code after the type checking in the compiled entry file, you will see the following code

 编译入口文件：src/cmd/compile/main.go -> gc.Main(archInit)

// Phase 4: Decide how to capture closed variables.（决定如何捕获闭包变量）
// This needs to run before escape analysis,
// because variables captured by value do not escape.（变量捕获应该在逃逸分析之前进行，因为值类型的变量捕获，不会进行逃逸分析）
    timings.Start("fe", "capturevars")
    for _, n := range xtop {
        if n.Op == ODCLFUNC && n.Func.Closure != nil { //函数需要是闭包类型
            Curfn = n
            capturevars(n)
        }
    }
    capturevarscomplete = true

From the above code and comments, we can get the following information:

1. Variable capture should be performed before escape analysis, because variable capture of value types will not perform escape analysis
2. Variable capture is for closure functions
3. The implementation of variable capture is mainly called: src/cmd/compile/internal/gc/closure.go→ capturevars

Below we will see the internal implementation of the method capturevars to understand some details of variable capture

Variable capture underlying implementation

After all type checking is done, capturevars will be called in a separate stage which decides whether each variable captured by the closure is captured by value or by reference

 func capturevars(xfunc *Node) {
    ......
    clo := xfunc.Func.Closure
    cvars := xfunc.Func.Cvars.Slice()
    out := cvars[:0]
    for _, v := range cvars {
        ......
        out = append(out, v)
        ......
        outer := v.Name.Param.Outer
        outermost := v.Name.Defn

        // out parameters will be assigned to implicitly upon return.
        if outermost.Class() != PPARAMOUT && !outermost.Name.Addrtaken() && !outermost.Name.Assigned() && v.Type.Width <= 128 {
            v.Name.SetByval(true)
        } else {
            outermost.Name.SetAddrtaken(true)
            outer = nod(OADDR, outer, nil)
        }
        ......
        outer = typecheck(outer, ctxExpr)
        clo.Func.Enter.Append(outer)
    }

    xfunc.Func.Cvars.Set(out)
    lineno = lno
}

The amount of code in this method is very small. The general content is that it will first obtain all variable nodes in the closure function, and then traverse these nodes. When it is determined that the variable that the closure needs to capture has not been modified, and the variable is less than 128 bytes, it will be considered a value reference . Later it will type-check externally referenced nodes

Summarize

This part is relatively simple, but very practical, especially for someone like me who has always been confused about how closures refer to external variables. The following escape analysis and closure rewriting have a certain connection with variable capture. I will mention it when I introduce the latter content.