Go every day a library of ants

Introduction

Handling a lot of concurrency is a big advantage of the Go language. The language has built-in convenient concurrent syntax, which can easily create many lightweight goroutine concurrent processing tasks. Compared to creating multiple threads, goroutines are lighter, consume less resources, switch faster, and have less overhead for threadless context switching. But limited by the total amount of resources, the number of goroutines that can be created in the system is also limited. By default, each goroutine occupies 8KB of memory. A machine with 8GB of memory can only create 8GB/8KB = 1000000 goroutines when it is full. What's more, the system also needs to reserve part of the memory to run daily management tasks. When go is running, it needs memory to run gc and process goroutines. Switch and so on. If the memory used exceeds the memory capacity of the machine, the system will use the swap area (swap), resulting in a rapid drop in performance. We can simply verify what happens when too many goroutines are created:

func main() {
  var wg sync.WaitGroup
  wg.Add(10000000)
  for i := 0; i < 10000000; i++ {
    go func() {
      time.Sleep(1 * time.Minute)
    }()
  }
  wg.Wait()
}

Running the above program on my machine (8G RAM) will report errno 1455 , that is, Out of Memory error, which is easy to understand. Run .

On the other hand, the management of goroutines is also a problem. A goroutine can only run to end by itself, and there is no external means to force j to end a goroutine. If a goroutine does not end on its own for some reason, a goroutine leak will occur. In addition, frequent creation of goroutines is also an overhead.

In view of the above reasons, naturally there is the same demand as the thread pool, namely the goroutine pool. A general goroutine pool automatically manages the life cycle of a goroutine, and can be created on demand and dynamically scaled down. Submit a task to the goroutine pool, and the goroutine pool will automatically schedule a certain goroutine to process it.

ants is one of the libraries that implements goroutine pool.

Quick to use

The code in this article uses Go Modules.

Create a directory and initialize:

$ mkdir ants && cd ants
$ go mod init github.com/darjun/go-daily-lib/ants

Install the ants library and use the v2 version:

$ go get -u github.com/panjf2000/ants/v2

We are going to implement a program that calculates the sum of a large number of integers. First create a basic task structure and implement its task execution method:

type Task struct {
  index int
  nums  []int
  sum   int
  wg    *sync.WaitGroup
}

func (t *Task) Do() {
  for _, num := range t.nums {
    t.sum += num
  }

  t.wg.Done()
}

It's very simple, just add up all the integers in a slice.

Then we create the goroutine pool. Note that the pool needs to be closed manually after use. Here we use defer close:

p, _ := ants.NewPoolWithFunc(10, taskFunc)
defer p.Release()

func taskFunc(data interface{}) {
  task := data.(*Task)
  task.Do()
  fmt.Printf("task:%d sum:%d\n", task.index, task.sum)
}

The above calls ants.NewPoolWithFunc() create a goroutine pool. The first parameter is the pool capacity, that is, there are up to 10 goroutines in the pool. The second parameter is the function that executes the task each time. When we call p.Invoke(data) , the ants pool will find a free one in its managed goroutine, let it execute the function taskFunc , and use data as a parameter.

Next, we simulate the data, do data segmentation, generate tasks, and hand them over to ants processing:

const (
  DataSize    = 10000
  DataPerTask = 100
)

nums := make([]int, DataSize, DataSize)
for i := range nums {
  nums[i] = rand.Intn(1000)
}

var wg sync.WaitGroup
wg.Add(DataSize / DataPerTask)
tasks := make([]*Task, 0, DataSize/DataPerTask)
for i := 0; i < DataSize/DataPerTask; i++ {
  task := &Task{
    index: i + 1,
    nums:  nums[i*DataPerTask : (i+1)*DataPerTask],
    wg:    &wg,
  }

  tasks = append(tasks, task)
  p.Invoke(task)
}

wg.Wait()
fmt.Printf("running goroutines: %d\n", ants.Running())

Randomly generate 10,000 integers, divide these integers into 100 parts, each with 100 parts, generate Task structure, and call p.Invoke(task) processing. wg.Wait() waits for the completion of the processing, and then outputs ants , which should be 0 at this time.

Finally, we summarize the results, verify the results, and compare them with the results obtained by direct addition:

var sum int
for _, task := range tasks {
  sum += task.sum
}

var expect int
for _, num := range nums {
  expect += num
}

fmt.Printf("finish all tasks, result is %d expect:%d\n", sum, expect)

run:

$ go run main.go
...
task:96 sum:53275
task:88 sum:50090
task:62 sum:57114
task:45 sum:48041
task:82 sum:45269
running goroutines: 0
finish all tasks, result is 5010172 expect:5010172

Indeed, after the task is completed, the number of running goroutines becomes 0. And we verified that the results are not biased. Also note that the goroutine pool is random and has nothing to do with the order of submission of tasks . We can also find this by running and printing the task ID above.

Function as task

ants supports submitting a function that does not accept any parameters as a task to the goroutine to run. Because it does not accept parameters, the function we submit either does not require external data, but only needs to process its own logic, otherwise, the required data must be passed in in some way, such as a closure.

The goroutine pool that submits the function as a task is ants.NewPool() , and it only accepts one parameter to indicate the capacity of the pool. Call the Submit() object to submit the task, and pass in a function that does not accept any parameters.

The first example can be rewritten. Add a task wrapper function, and use the parameters required by the task as the parameters of the wrapper function. The wrapper function returns the actual task function, and the task function can access the data it needs through the closure:

type taskFunc func()

func taskFuncWrapper(nums []int, i int, sum *int, wg *sync.WaitGroup) taskFunc {
  return func() {
    for _, num := range nums[i*DataPerTask : (i+1)*DataPerTask] {
      *sum += num
    }

    fmt.Printf("task:%d sum:%d\n", i+1, *sum)
    wg.Done()
  }
}

Call ants.NewPool(10) create a goroutine pool. Similarly, the pool needs to be released when it is used up. Here, defer is used:

p, _ := ants.NewPool(10)
defer p.Release()

Generate simulation data and split tasks. Submit the task to the ants pool for execution. Here, use the taskFuncWrapper() wrapper function to generate a specific task, and then call p.Submit() submit:

nums := make([]int, DataSize, DataSize)
for i := range nums {
  nums[i] = rand.Intn(1000)
}

var wg sync.WaitGroup
wg.Add(DataSize / DataPerTask)
partSums := make([]int, DataSize/DataPerTask, DataSize/DataPerTask)
for i := 0; i < DataSize/DataPerTask; i++ {
  p.Submit(taskFuncWrapper(nums, i, &partSums[i], &wg))
}
wg.Wait()

Summarize the results and verify:

var sum int
for _, partSum := range partSums {
  sum += partSum
}

var expect int
for _, num := range nums {
  expect += num
}
fmt.Printf("running goroutines: %d\n", ants.Running())
fmt.Printf("finish all tasks, result is %d expect is %d\n", sum, expect)

The function of this program is exactly the same as the original one.

Implementation process

There is an execution flow chart in the GitHub repository. I redrawn it:

The execution process is as follows:

• Initialize the goroutine pool;

• Submit the task to the goroutine pool and check if there are free goroutines:

  • Yes, get free goroutine

  • None, check whether the number of goroutines in the pool has reached the upper limit of the pool capacity:

    • The upper limit has been reached, check whether the goroutine pool is non-blocking:

      • Non-blocking, directly return nil indicate execution failure
      • Blocking, waiting for the goroutine to be free
    • The upper limit is not reached, create a new goroutine to handle the task
• After the task is processed, return the goroutine to the pool for the next task

Options

ants provides some options to customize the behavior of the goroutine pool. Option to use Options structure definition:

// src/github.com/panjf2000/ants/options.go
type Options struct {
  ExpiryDuration time.Duration
  PreAlloc bool
  MaxBlockingTasks int
  Nonblocking bool
  PanicHandler func(interface{})
  Logger Logger
}

The meaning of each option is as follows:

• ExpiryDuration : Expiration time. Indicates how long the goroutine will be recycled ants
• PreAlloc : Pre-allocation. After calling NewPool()/NewPoolWithFunc() , pre-allocate worker (a structure that manages a working goroutine) slice. Moreover, the use of pre-allocation will directly affect the structure of the worker See the source code below
• MaxBlockingTasks : The maximum number of blocked tasks. That is, the number of goroutines in the pool has reached the capacity of the pool, and all goroutines are processing busy. At this time, the incoming tasks will wait on the blocking list. This option sets the maximum length of the list. After the number of blocked tasks reaches this value, the subsequent task submission directly returns to failure
• Nonblocking : Whether the pool is blocked, it is blocked by default. When submitting a task, if the ants upper limit and are all busy, the blocked pool will wait for the blocked list added by the task (of course, limited by the length of the blocked list, see the previous option). Non-blocking pool returns directly to failure
• PanicHandler : panic processing. When encountering panic, the processing function set here will be called
• Logger : Specify the logger

NewPool() part of the source code:

if p.options.PreAlloc {
  if size == -1 {
    return nil, ErrInvalidPreAllocSize
  }
  p.workers = newWorkerArray(loopQueueType, size)
} else {
  p.workers = newWorkerArray(stackType, 0)
}

When using the pre-allocated, creating loopQueueType types of structures, and vice versa to create stackType type. This is ants two management defined worker data structure.

ants defines some With* functions to set these options:

func WithOptions(options Options) Option {
  return func(opts *Options) {
    *opts = options
  }
}

func WithExpiryDuration(expiryDuration time.Duration) Option {
  return func(opts *Options) {
    opts.ExpiryDuration = expiryDuration
  }
}

func WithPreAlloc(preAlloc bool) Option {
  return func(opts *Options) {
    opts.PreAlloc = preAlloc
  }
}

func WithMaxBlockingTasks(maxBlockingTasks int) Option {
  return func(opts *Options) {
    opts.MaxBlockingTasks = maxBlockingTasks
  }
}

func WithNonblocking(nonblocking bool) Option {
  return func(opts *Options) {
    opts.Nonblocking = nonblocking
  }
}

func WithPanicHandler(panicHandler func(interface{})) Option {
  return func(opts *Options) {
    opts.PanicHandler = panicHandler
  }
}

func WithLogger(logger Logger) Option {
  return func(opts *Options) {
    opts.Logger = logger
  }
}

A very common mode in the Go language is used here. I call it the option mode. It is very convenient to construct objects with a large number of parameters, and most of them have default values or generally do not need to be explicitly set.

Let's verify a few options.

Maximum waiting queue length

ants setting the capacity of the 060beba15ba109 pool, if all goroutines are processing tasks. At this time, the submitted task will enter the waiting queue by default. WithMaxBlockingTasks(maxBlockingTasks int) can set the maximum length of the waiting queue. If the length exceeds this length, submit the task and return an error directly:

func wrapper(i int, wg *sync.WaitGroup) func() {
  return func() {
    fmt.Printf("hello from task:%d\n", i)
    time.Sleep(1 * time.Second)
    wg.Done()
  }
}

func main() {
  p, _ := ants.NewPool(4, ants.WithMaxBlockingTasks(2))
  defer p.Release()

  var wg sync.WaitGroup
  wg.Add(8)
  for i := 1; i <= 8; i++ {
    go func(i int) {
      err := p.Submit(wrapper(i, &wg))
      if err != nil {
        fmt.Printf("task:%d err:%v\n", i, err)
        wg.Done()
      }
    }(i)
  }

  wg.Wait()
}

In the above code, we set the capacity of the goroutine pool to 4 and the maximum blocking queue length to 2. Then a for submits 8 tasks, and the expected result is: 4 tasks are executing, 2 tasks are waiting, and 2 tasks fail to be submitted. operation result:

hello from task:8
hello from task:5
hello from task:4
hello from task:6
task:7 err:too many goroutines blocked on submit or Nonblocking is set
task:3 err:too many goroutines blocked on submit or Nonblocking is set
hello from task:1
hello from task:2

We see that the submission of the task failed, print too many goroutines blocked ... .

There are 4 points to note in the code:

• submission tasks must be carried out in parallel. If it is a serial submission, when the fifth task is submitted, because there is no free goroutine in the pool to process the task, the Submit() method will be blocked, and subsequent tasks cannot be submitted. It will not achieve the purpose of
• Because the task may fail to be submitted, the failed task will not be actually executed, so the wg.Done() will actually be less than 8. Therefore, we need to call wg.Done() once in the err != nil branch. Otherwise wg.Wait() will always block
• In order to avoid task execution too fast, the goroutine is vacated, and no phenomenon is observed. For each task, I use time.Sleep(1 * time.Second) sleep 1s
• Because the execution order between goroutines is not explicitly synchronized, the order of each execution is uncertain

For simplicity, Submit() method in the previous example has been ignored. Don't ignore it in actual development.

Non-blocking

ants pool is blocked by default, we can use WithNonblocking(nonblocking bool) set it to non-blocking. In the non-blocking ants pool, when all goroutines are processing tasks, submitting a new task will directly return an error:

func main() {
  p, _ := ants.NewPool(2, ants.WithNonblocking(true))
  defer p.Release()

  var wg sync.WaitGroup
  wg.Add(3)
  for i := 1; i <= 3; i++ {
    err := p.Submit(wrapper(i, &wg))
    if err != nil {
      fmt.Printf("task:%d err:%v\n", i, err)
      wg.Done()
    }
  }

  wg.Wait()
}

wrapper() function in the previous example, the ants pool capacity is set to 2. Submit 3 tasks in a row, expect that the first two tasks will be executed normally, and an error will be returned when the third task is submitted:

hello from task:2
task:3 err:too many goroutines blocked on submit or Nonblocking is set
hello from task:1

panic processor

A robust library will not ignore error handling, especially crash-related errors. In the Go language, it is panic, also known as runtime panic. Serious errors that occur during program operation, such as index out of range, null pointer dereference, etc., will trigger panic. If the panic is not processed, the program will exit unexpectedly, which may cause serious consequences of data loss.

ants If goroutine occurred while performing the task panic , will terminate the current task execution, an error will occur stack output to os.Stderr . Note that the goroutine will still be put back into the pool and can be taken out to perform a new task next time.

func wrapper(i int, wg *sync.WaitGroup) func() {
  return func() {
    fmt.Printf("hello from task:%d\n", i)
    if i%2 == 0 {
      panic(fmt.Sprintf("panic from task:%d", i))
    }
    wg.Done()
  }
}

func main() {
  p, _ := ants.NewPool(2)
  defer p.Release()

  var wg sync.WaitGroup
  wg.Add(3)
  for i := 1; i <= 2; i++ {
    p.Submit(wrapper(i, &wg))
  }

  time.Sleep(1 * time.Second)
  p.Submit(wrapper(3, &wg))
  p.Submit(wrapper(5, &wg))
  wg.Wait()
}

We let an even number of tasks trigger panic . Submit two tasks, the second task will definitely trigger panic . After triggering panic , we can continue to submit tasks 3 and 5. Note that there is no 4, and submitting task 4 will still trigger panic .

The above program needs to pay attention to 2 points:

• task function wg.Done() is panic After the method, if the trigger panic , other normal functions of logic will not continue to be executed. So although we wg.Add(3) , we submitted a total of 4 tasks, one of which triggered panic , and wg.Done() did not execute correctly. In actual development, we generally use the defer statement to ensure that wg.Done() will be executed
• After the for loop, I added a line of code time.Sleep(1 * time.Second) . Without this line, the subsequent two Submit() methods can be executed directly, which may cause the task to be completed soon, and wg.Wait() returns directly. At this time, the stack of panic . You can try to comment out this line of code and run it to see the result

In addition to the default panic processor provided by ants WithPanicHandler(paincHandler func(interface{})) specify our own panic processor. The parameter of the processor is the value panic

func panicHandler(err interface{}) {
  fmt.Fprintln(os.Stderr, err)
}

p, _ := ants.NewPool(2, ants.WithPanicHandler(panicHandler))
defer p.Release()

The rest of the code is exactly the same as above. panic panicHandler triggered and it will be executed. run:

hello from task:2
panic from task:2
hello from task:1
hello from task:5
hello from task:3

See that the string passed to the panic function is output (the second line of output).

Default pool

For ease of use, many Go libraries like to provide a default implementation of their core functional types. It can be called directly through the interface provided by the library. For example, net/http , such as ants . A default pool is defined in the ants MaxInt32 . All methods of the goroutine pool can be directly accessed through the ants package:

// src/github.com/panjf2000/ants/ants.go
defaultAntsPool, _ = NewPool(DefaultAntsPoolSize)

func Submit(task func()) error {
  return defaultAntsPool.Submit(task)
}

func Running() int {
  return defaultAntsPool.Running()
}

func Cap() int {
  return defaultAntsPool.Cap()
}

func Free() int {
  return defaultAntsPool.Free()
}

func Release() {
  defaultAntsPool.Release()
}

func Reboot() {
  defaultAntsPool.Reboot()
}

Use directly:

func main() {
  defer ants.Release()

  var wg sync.WaitGroup
  wg.Add(2)
  for i := 1; i <= 2; i++ {
    ants.Submit(wrapper(i, &wg))
  }
  wg.Wait()
}

The default pool also requires Release() .

to sum up

This article introduces the origin of the goroutine pool, and introduces the basic usage method and some details based on the ants ants not much, and the core code without testing is only about 1k lines. It is recommended that those who have time and are interested in reading it in depth.

If you find a fun and useful Go language library, welcome to submit an issue on the Go Daily Library GitHub😄

reference

1. ants GitHub：github.com/panjf2000/ants
2. Go daily library GitHub: https://github.com/darjun/go-daily-lib

I

My blog: https://darjun.github.io

Welcome to follow my WeChat public account [GoUpUp], learn together and make progress together~