Uncomfortable, producing Go timer.After the pain of memory leak!

search [160bef95c09394 brain into the fried fish ] Follow this fried fish with liver fried. This article GitHub github.com/eddycjy/blog has been included, and there are my series of articles, materials and open source Go books.

Hello everyone, I am fried fish.

A few days ago, I shared an article about Go timer source code analysis "Go timer that is difficult to control. This article will take you through the mystery of timers."

In the comment area, some friends mentioned the classic timer.After leakage problem, I hope I can talk about it, this is a big "pit" that I must know.

Today’s article Fried Fish will take everyone to discuss this issue.

timer.After

Today is the male protagonist is the After method provided by the Go standard library time. The function signature is as follows:

func After(d Duration) <-chan Time 

This method can actively return a channel message of type time.Time after a certain period of time (according to the incoming Duration).

In common scenarios, we will develop some timer-related functions based on this method. Examples are as follows:

func main() {
    ch := make(chan string)
    go func() {
        time.Sleep(time.Second * 3)
        ch <- "脑子进煎鱼了"
    }()

    select {
    case _ = <-ch:
    case <-time.After(time.Second * 1):
        fmt.Println("煎鱼出去了，超时了！！！")
    }
}

After running for 1 second, output the result:

煎鱼出去了，超时了！！！

time.After method after running for 1 second, and outputs the timeout result.

Where is the pit

Judging from the examples, it seems very normal, and there is no "pit". Could it be a timer.After shot of the 060bef95c094f0 method?

Let's look at another example that doesn't seem to be problematic. This is often seen in Go projects, but everyone doesn't pay much attention.

code show as below:

func main() {
    ch := make(chan int, 10)
    go func() {
        in := 1
        for {
            in++
            ch <- in
        }
    }()
    
    for {
        select {
        case _ = <-ch:
            // do something...
            continue
        case <-time.After(3 * time.Minute):
            fmt.Printf("现在是：%d，我脑子进煎鱼了！", time.Now().Unix())
        }
    }
}

In the above code, we constructed a classic processing mode of for+select+channel

While select+case call a time.After way to do overtime control, to avoid channel blocking while waiting too long, causing other problems.

It doesn't seem to be a problem, but take a closer look. After running for a period of time, rudely use the top command to take a look:

The memory usage of my Go project has reached as high as 10+GB, and it continues to grow, which is terrible.

After the set timeout period is reached, the memory usage of the Go project does not seem to fall back for a while. What happened?

why

With a dumb face of fried fish, I silently took out the PProf that I had already buried. This is the strongest performance analysis tool in the Go language. It is specially spent in my "Go Language Programming Journey". The chapters will be explained extensively.

In the Go language, PProf is a tool for visualizing and analyzing performance analysis data. PProf uses profile.proto to read the collection of analysis samples and generate reports to visualize and help analyze the data (text and graphic reports are supported).

We directly use go tool pprof analyze the function memory application in the Go project, as shown in the following figure:

From the analysis of the figure, it can be found that time.After is constantly called, which leads to the continuous creation and memory application of the time.NerTimer

This is very strange, because there are only a few lines of code associated with time in our Go project:

func main() {
    ...
    for {
        select {
        ...
        case <-time.After(3 * time.Minute):
            fmt.Printf("现在是：%d，我脑子进煎鱼了！", time.Now().Unix())
        }
    }
}

Since the Demo is small enough, we believe this is the problem code, but what is the reason?

The reason is that the for + select , plus time.After will cause memory leaks. Because for will call the select statement when looping, so every time select is executed, a brand new timer (Timer) will be reinitialized.

Our timer is triggered to perform certain things after 3 minutes, but the point is that after the timer is activated, it is found that select is no reference relationship with 060bef95c0b249, so it is reasonable to be cleaned up by the GC Because no one needs "me" anymore.

time.After is yet to come. The abandoned 060bef95c0b290 timing task is still waiting in the time heap to be triggered. Before the timing task expires, it will not be cleared by the GC.

But unfortunately, it will never expire, which is why the memory of our Go project keeps soaring. In fact, it is the memory orphans generated by time.After

Solution

Now that we know that the root cause of the problem is the repeated creation of time.After and the closed loop of the release cannot be completed, then there is a solution.

The improved code is as follows:

func main() {
    timer := time.NewTimer(3 * time.Minute)
    defer timer.Stop()
    
    ...
    for {
        select {
        ...
        case <-timer.C:
            fmt.Printf("现在是：%d，我脑子进煎鱼了！", time.Now().Unix())
        }
    }
}

After a period of fishing, use PProf to collect and view:

The indicators of the Go process are normal, and the memory leak problem is solved intact.

to sum up

In today's article, we introduced the time , and at the same time, we have reproduced and analyzed the memory leak caused by the improper use time.After

The root cause lies in the leakage caused by the processing mechanism of the Go language time heap and for + select + time.After

It suddenly occurred to me that a friend of mine had seen similar code in the company, stepped on this pit in production, and was arrested by an alarm in the middle of the night...

I don't know if you have encountered similar problems in your daily work. Welcome to comment and exchange in the message area.

The article is continuously updated, and you can read it on [My brain is fried fish], and reply [160bef95c0b546 000 ] I have prepared the first-line interview algorithm questions and information; this article GitHub github.com/edf95c0b54e , Welcome Star to urge you to update.