9 pictures to easily understand the Go memory management unit

Guided reading

If you want to deeply understand the memory management implementation of the Go language, it is inevitable to avoid the "Go memory management unit mspan ", today we will go deep and unravel the "Go memory management unit mspan " the mystery.

The specific concepts also included in this article are as follows:

• The concept of page
• The concept of mspan
• The concept of object
• The concept of FreeList
• The concept of sizeclass
• The concept of spanclass

text

Before introducing the Go memory management unit mspan , you need to look at the concept of page , because mspan is composed of N and continuous page composition.

The concept of page

The operating system manages memory according to page , and the Go language also manages memory according to page . 1page is 8KB, which is consistent with the operating system, as shown in the following figure:

The Go memory management unit mspan is usually composed of N and contiguous page , as shown in the following figure:

mspan consists of page

• mspan can be composed of 1 page
• mspan can also be composed of 2 consecutive page
• mspan can also be composed of 3 consecutive page
• mspan can be composed of N consecutive page

The left half of the above figure is the key field of the structure of mspan , of which npages represents this mspan is composed of several consecutive page Composition.

In addition, a linked list can also be formed between mspan and mspan , as shown in the following figure

mspan can form a linked list

It should be noted here that only npages with the same value mspan can form a linked list . As shown in the figure above, the specific reasons will be discussed below.

Seeing this, do you think that Go allocates memory by page page 8KB as the smallest unit?

Answer: Of course not, if so it will result in low memory usage. The Go language memory manager will mspan into smaller granular units object . As shown below:

object和---e0a3e29d4e4cc1b461669ee8380e053c object ，大家这里肯定会想到是LinkedList ，实际上并不是，因为LinkedList The pointer will also occupy 8B memory. As a memory manager, this part of the memory will be wasted in vain, so the data structure usually used here is FreeList .

What is FreeList?

FreeList is essentially a LinkedList , and the difference between LinkedList :

• FreeList does not have the Next attribute, so instead of using the Next attribute to store the value of the pointer to the next node.
• FreeList "equivalent to using the first 8 bytes of Value " (in fact, the first 8 bytes of the entire memory) to store the pointer of the next node.
• For the allocated node, the entire memory space of the node can be overwritten (the value of the pointer can be overwritten)

As shown below:

So: FreeList A node is at least 8 bytes Byte

 备注：因为要存指针，指针的大小为8字节，为什么？可以参考之前文章《64位平台下，指针自身的大小为什么是8字节？》

The Go memory management unit mspan was disassembled as object as shown below:

Here comes the question again, what is the specific size of object and how is it determined?

Answer: It is determined by sizeclass .

What is sizeclass ?

sizeclass映射列表，实际是一个数组类型[68]uint16 ，它的object的大小，除此之外， mspan composition of several pages is also determined by the value of sizeclass . sizeclass The specific rules of the mapping list are as follows:

 // 文件位置：`src/runtime/sizeclasses.g`
// 索引0位置被保留使用，具体使用位置后续会讲。

如上文所述，`object`之间采用freelist数据结构构成链表，指针为8Byte所以最小的object大小为8Byte

字段解释：
class: sizeclass值 
bytes/obj: 该`mspan`拆分object大小
bytes/span: 该`mspan`是由几pages组成
objects: 该`mspan`共计包含的object数量
tail waste: 该`mspan`拆分为object之后，mspan剩余末尾浪费的内存

// class  bytes/obj  bytes/span  objects     tail waste  max waste
//
//     1          8        8192     1024           0     87.50%
//     2         16        8192      512           0     43.75%
//     3         24        8192      341           8     29.24%
//     4         32        8192      256           0     21.88%
//     5         48        8192      170          32     31.52%
//     6         64        8192      128           0     23.44%
// 略...
//    62      20480       40960        2           0      6.87%
//    63      21760       65536        3         256      6.25%
//    64      24576       24576        1           0     11.45%
//    65      27264       81920        3         128     10.00%
//    66      28672       57344        2           0      4.91%
//    67      32768       32768        1           0     12.50%

sizeclass value	object size	Composed of several pages
0	reserve	1page
1	8Byte	1pages
2	16Byte	1page
3	24Byte	1page
...	...	...
67	32KB	4pages

So mspan the structure, as long as you maintain a field of sizeclass , you can know the size and quantity of the mspan in object However, in fact, this field is not sizeclass , but spanclass , as shown in the following figure:

So, here comes the problem again.

What is spanclass ?

---

Actually the Go memory management unit mspan is divided into two categories:

• The first category: mspan for garbage collection scanning, referred to as scan
• The second category: mspan without garbage collection scanning, referred to as noscan

So not all Go memory management units mspan will be scanned by garbage collection . In order to distinguish the two types mspan , the Go language puts the type identifier and the value of the above sizeclass in the same field, as follows:

• sizeclass value is shifted left one place: sizeclass << 1

• sizeclass The last digit of the value is stored in the type

  • The last bit is 1: it does not need garbage collection scanning mspan
  • The last bit is 0: it needs garbage collection scanning mspan

The diagram is as follows:

Summarize

mspan split object summary

Here we take the decimal value of spanclass mspan as an example:

spanclass Decimal value is 7
Available, spanclass binary is 0000 0111
Available, sizeclass is 7>>1 : binary 0000 0011 , decimal 3
Available, mspan consists of 1 page , a total of 8KB (8192Byte)
Available, object size is 24Byte
Available, mspan including 341 in total object
Available, mspan wasting 8Byte at the tail

The specific diagram is as follows:

mspan Summary of key fields

Select several important fields of mspan , as shown below:

field name	explain
next, prev, list	mspan can form a linked list
startAddr	mspan starting position of memory, N consecutive page starting position of memory
npages	mspan consists of several page
freeindex	Idle object starting position of linked list
nelems	How many in total object
spanclass	Determines the size of object , and whether the current mspan needs a garbage collection scan
...	...

View more content in the "Go Language Easy and Advanced" series

Link http://tigerb.cn/go/#/kernal/