The right assistant of Hongmeng Light Kernel: Take you to master 4 kinds of memory debugging methods

Abstract: The memory debugging method is designed to assist in locating dynamic memory related problems. It provides three debugging methods: memory pool information statistics, memory leak detection, and memory stepping detection.

This article is shared from the Huawei Cloud Community " Light Kernel-Memory -Memory Information Statistics 1613971b270035", author: zhushy.

The memory debugging method is designed to assist in locating dynamic memory related problems, providing basic dynamic memory pool information statistics methods, presenting users with information such as memory pool waterline, fragmentation rate, etc.; providing memory leak detection methods to facilitate users to accurately locate memory leaks The line of code can also assist in the analysis of the memory usage of each module of the system; it provides a memory stepping detection method, which can assist in locating the scene of crossing the boundary and stepping on the memory.

1. Memory information statistics

Memory information includes memory pool size, memory usage, remaining memory size, maximum free memory, memory waterline, statistics on the number of memory nodes, fragmentation rate, etc.

• Memory waterline: the maximum usage of the memory pool, the waterline value will be updated every time it is applied for and released, and the actual business can optimize the memory pool size based on this value;
• Fragmentation rate: a measure of the degree of fragmentation of the memory pool. A high fragmentation rate means that there is a lot of remaining memory in the memory pool, but the maximum free memory block is very small. It can be measured by the formula (fragment=100-maximum free memory block size/remaining memory size) ；
• Other parameters: Scan the node information of the memory pool through the call interface of the memory management module, and collect relevant information.

1. Function configuration

LOSCFG_MEM_WATERLINE: switch macro, open by default; if this function is closed, define this macro as 0 in target_config.h. If you need to get the memory waterline, you need to turn on this configuration.

2. Development guidance

Key structure introduction:

typedef struct {
    UINT32 totalUsedSize;       // 内存池的内存使用量
    UINT32 totalFreeSize;       // 内存池的剩余内存大小
    UINT32 maxFreeNodeSize;     // 内存池的最大空闲内存块大小
    UINT32 usedNodeNum;         // 内存池的非空闲内存块个数
    UINT32 freeNodeNum;         // 内存池的空闲内存块个数
#if (LOSCFG_MEM_WATERLINE == 1) // 默认打开，如需关闭，在target_config.h中将该宏设置为0
    UINT32 usageWaterLine;      // 内存池的水线值
#endif
} LOS_MEM_POOL_STATUS;

• Memory waterline acquisition
  Call the LOS_MemInfoGet interface, the first parameter is the first address of the memory pool, the second parameter is a handle of type LOS_MEM_POOL_STATUS, and the field usageWaterLine is the waterline value.
• Memory fragmentation rate calculation
  Also call the LOS_MemInfoGet interface to obtain the remaining memory size and the maximum free memory block size of the memory pool, and then obtain the dynamic memory pool fragmentation rate at this time according to the formula (fragment=100-maximum free memory block size/remaining memory size).

3. Programming example

This example implements the following functions:

1. Create a monitoring thread to obtain information about the memory pool;
2. Call the LOS_MemInfoGet interface to obtain the basic information of the memory pool;
3. Use the formula to calculate the utilization rate and fragmentation rate.

The code is implemented as follows:

#include <stdio.h>
#include <string.h>
#include "los_task.h"
#include "los_memory.h"
#include "los_config.h"

void MemInfoTaskFunc(void)
{
    LOS_MEM_POOL_STATUS poolStatus = {0};
    LOS_MemInfoGet(m_aucSysMem0, &poolStatus);
    /* 算出内存池当前的碎片率百分比 */
    unsigned char fragment = 100 - poolStatus.maxFreeNodeSize * 100 / poolStatus.totalFreeSize;
    /* 算出内存池当前的使用率百分比 */
    unsigned char usage = LOS_MemTotalUsedGet(m_aucSysMem0) * 100 / LOS_MemPoolSizeGet(m_aucSysMem0);
    printf("usage = %d, fragment = %d, maxFreeSize = %d, totalFreeSize = %d, waterLine = %d\n", usage, fragment, poolStatus.maxFreeNodeSize, 
           poolStatus.totalFreeSize, poolStatus.usageWaterLine);
}

int MemTest(void)
{
    unsigned int ret;
    unsigned int taskID;
    TSK_INIT_PARAM_S taskStatus = {0};
    taskStatus.pfnTaskEntry = (TSK_ENTRY_FUNC)MemInfoTaskFunc;
    taskStatus.uwStackSize  = 0x1000;
    taskStatus.pcName       = "memInfo";
    taskStatus.usTaskPrio   = 10;
    ret = LOS_TaskCreate(&taskID, &taskStatus);
    if (ret != LOS_OK) {
        printf("task create failed\n");
        return -1;
    }
    return 0;
}

The output of compiling and running is as follows:

usage = 22, fragment = 3, maxFreeSize = 49056, totalFreeSize = 50132, waterLine = 1414

Two, memory leak detection mechanism

The memory leak detection mechanism is an optional function of the kernel to assist in locating dynamic memory leaks. When this function is enabled, the dynamic memory mechanism will automatically record the function call relationship when applying for memory (hereinafter referred to as LR). If there is a leak, you can use the recorded information to find the memory application for further confirmation.

1. Function configuration

• LOSCFG_MEM_LEAKCHECK: switch macro, closed by default; if this function is turned on, define this macro as 1 in target_config.h.
• LOSCFG_MEM_RECORD_LR_CNT: The number of LR layers recorded, the default is 3 layers; each layer of LR consumes sizeof(void *) bytes of memory.
• LOSCFG_MEM_OMIT_LR_CNT: The number of LR layers that are ignored, the default is 4 layers, that is, the recording starts from calling the function of LOS_MemAlloc, which can be adjusted according to the actual situation. Why do you need this configuration? There are 3 reasons as follows:
• There are also function calls inside the LOS_MemAlloc interface;
• The external may have encapsulation on the LOS_MemAlloc interface;
• The number of LR layers configured by LOSCFG_MEM_RECORD_LR_CNT is limited;
  Configure this macro correctly and ignore the number of invalid LR layers, then you can record the number of valid LR layers and save memory consumption.

2. Development guidance

2.1 Development process

This debugging function can analyze whether there is a memory leak in the key code logic. Enabling this function will record LR information every time you apply for memory. Before and after the code segment that needs to be detected, call the LOS_MemUsedNodeShow interface, and print all the node information that has been used in the specified memory pool each time. Compare the two node information before and after, and the new node information is the memory node that is suspected of leaking. Through LR, you can find the code location of the specific application to further confirm whether it is leaked.

The format of the node information output by calling the LOS_MemUsedNodeShow interface is as follows: every 1 line is a node information; the first column is the node address, you can use GDB and other means to view the complete information of the node based on this address; the second column is the size of the node, which is equal to the size of the node header +Data field size; the 3rd~5th column is the function call relation LR address. According to this value, combined with the assembly file, you can view the specific application location of the node.

node        size   LR[0]      LR[1]       LR[2]  
0x10017320: 0x528 0x9b004eba  0x9b004f60  0x9b005002 
0x10017848: 0xe0  0x9b02c24e  0x9b02c246  0x9b008ef0 
0x10017928: 0x50  0x9b008ed0  0x9b068902  0x9b0687c4 
0x10017978: 0x24  0x9b008ed0  0x9b068924  0x9b0687c4
0x1001799c: 0x30  0x9b02c24e  0x9b02c246  0x9b008ef0 
0x100179cc: 0x5c  0x9b02c24e  0x9b02c246  0x9b008ef0 

Note: will affect the performance of memory applications, and each memory node will record the LR address, and the memory overhead will also increase.

2.2 Programming example

This example implements the following functions: build a memory leak code segment.

1. Call LOS_MemUsedNodeShow interface, output all node information and print;
2. Apply for memory, but not release, simulating memory leak;
3. Call the LOS_MemUsedNodeShow interface again to output all node information and print;
4. Compare the two logs to get the leaked node information;
5. Through the LR address, find out the location of the leaked code;

2.3 Sample code

The code is implemented as follows:

#include <stdio.h>
#include <string.h>
#include "los_memory.h"
#include "los_config.h"

void MemLeakTest(void)
{
    LOS_MemUsedNodeShow(LOSCFG_SYS_HEAP_ADDR);
    void *ptr1 = LOS_MemAlloc(LOSCFG_SYS_HEAP_ADDR, 8);
    void *ptr2 = LOS_MemAlloc(LOSCFG_SYS_HEAP_ADDR, 8);
    LOS_MemUsedNodeShow(LOSCFG_SYS_HEAP_ADDR);
}

2.4 Result verification

The compile and run output log is as follows:

node         size   LR[0]       LR[1]       LR[2]   
0x20001b04:  0x24   0x08001a10  0x080035ce  0x080028fc 
0x20002058:  0x40   0x08002fe8  0x08003626  0x080028fc 
0x200022ac:  0x40   0x08000e0c  0x08000e56  0x0800359e 
0x20002594:  0x120  0x08000e0c  0x08000e56  0x08000c8a 
0x20002aac:  0x56   0x08000e0c  0x08000e56  0x08004220 

node         size   LR[0]       LR[1]       LR[2]   
0x20001b04:  0x24   0x08001a10  0x080035ce  0x080028fc 
0x20002058:  0x40   0x08002fe8  0x08003626  0x080028fc 
0x200022ac:  0x40   0x08000e0c  0x08000e56  0x0800359e 
0x20002594:  0x120  0x08000e0c  0x08000e56  0x08000c8a 
0x20002aac:  0x56   0x08000e0c  0x08000e56  0x08004220 
0x20003ac4:  0x1d   0x08001458  0x080014e0  0x080041e6 
0x20003ae0:  0x1d   0x080041ee  0x08000cc2  0x00000000

Comparing the two logs, the difference is as follows. These memory nodes are the memory blocks that are suspected of leaking:

0x20003ac4:  0x1d   0x08001458  0x080014e0  0x080041e6 
0x20003ae0:  0x1d   0x080041ee  0x08000cc2  0x00000000 

Part of the assembly file is as follows:

              MemLeakTest:
 0x80041d4: 0xb510         PUSH     {R4, LR}
 0x80041d6: 0x4ca8         LDR.N    R4, [PC, #0x2a0]       ; g_memStart
 0x80041d8: 0x0020         MOVS     R0, R4
 0x80041da: 0xf7fd 0xf93e  BL       LOS_MemUsedNodeShow    ; 0x800145a
 0x80041de: 0x2108         MOVS     R1, #8
 0x80041e0: 0x0020         MOVS     R0, R4
 0x80041e2: 0xf7fd 0xfbd9  BL       LOS_MemAlloc           ; 0x8001998
 0x80041e6: 0x2108         MOVS     R1, #8
 0x80041e8: 0x0020         MOVS     R0, R4
 0x80041ea: 0xf7fd 0xfbd5  BL       LOS_MemAlloc           ; 0x8001998
 0x80041ee: 0x0020         MOVS     R0, R4
 0x80041f0: 0xf7fd 0xf933  BL       LOS_MemUsedNodeShow    ; 0x800145a
 0x80041f4: 0xbd10         POP      {R4, PC}
 0x80041f6: 0x0000         MOVS     R0, R0

Among them, by searching for 0x080041ee, you can find that the memory node is applied for in the MemLeakTest interface and has not been released.

Third, step on the memory detection mechanism

As an optional function of the kernel, the stepped memory detection mechanism is used to detect the integrity of the dynamic memory pool. Through this mechanism, it is possible to discover in time whether a memory stepping problem occurs in the memory pool, and give an error message, which facilitates timely discovery of system problems, improves problem solving efficiency, and reduces problem location costs.

1. Function configuration

LOSCFG_BASE_MEM_NODE_INTEGRITY_CHECK: switch macro, closed by default; if this function is turned on, define this macro as 1 in target_config.h.

1. When this function is enabled, the integrity of the memory pool will be checked in real time every time memory is requested.
2. If you do not enable this function, you can also call the LOS_MemIntegrityCheck interface to detect, but each time you apply for memory, the memory integrity will not be detected in real time, and because the node head does not have a devil number (only available when it is turned on, saving memory), the accuracy of the detection is also It will be reduced accordingly, but has no effect on the performance of the system, so switch this function according to the actual situation.

Since this function will only detect which memory node is destroyed, and give the previous node information (because the memory distribution is continuous, the current node is most likely to be destroyed by the previous node). If you want to further confirm where the previous node applied for, you need to turn on the memory leak detection function and use the LR record to assist in positioning.

Note: When turns on this function, the node header has an extra devil number field, which will increase the size of the node header. Due to the real-time integrity detection, the performance impact is greater; if the performance is sensitive, you can disable this function and use the LOS_MemIntegrityCheck interface to detect.

2. Development guidance

2.1 Development process

By calling the LOS_MemIntegrityCheck interface, check whether the memory pool has stepped on memory. If there is no stepping on memory, the interface returns 0 and no log output; if there is a stepping on memory problem, the relevant log will be output. See the result output of the programming example below for details.

2.2 Programming example

This example implements the following functions:

1. Apply for two physically contiguous memory blocks;
2. Construct out-of-bounds access through memset, stepping on the first 4 bytes of the next node;
3. Call LOS_MemIntegrityCheck to detect whether memory stepping occurs.

2.3 Sample code

The code is implemented as follows:

#include <stdio.h>
#include <string.h>
#include "los_memory.h"
#include "los_config.h"

void MemIntegrityTest(void)
{
    /* 申请两个物理连续的内存块 */
    void *ptr1 = LOS_MemAlloc(LOSCFG_SYS_HEAP_ADDR, 8);
    void *ptr2 = LOS_MemAlloc(LOSCFG_SYS_HEAP_ADDR, 8);
    /* 第一个节点内存块大小是8字节，那么12字节的清零，会踩到第二个内存节点的节点头，构造踩内存场景 */
    memset(ptr1, 0, 8 + 4);
    LOS_MemIntegrityCheck(LOSCFG_SYS_HEAP_ADDR);
}

2.4 Result verification

The compile and run output log is as follows:

[ERR][OsMemMagicCheckPrint], 2028, memory check error!
memory used but magic num wrong, magic num = 0x00000000   /* 提示信息，检测到哪个字段被破坏了，用例构造了将下个节点的头4个字节清零，即魔鬼数字字段 */

 broken node head: 0x20003af0  0x00000000  0x80000020, prev node head: 0x20002ad4  0xabcddcba  0x80000020   
/* 被破坏节点和其前节点关键字段信息，分别为其前节点地址、节点的魔鬼数字、节点的sizeAndFlag；可以看出被破坏节点的魔鬼数字字段被清零，符合用例场景 */

 broken node head LR info:  /* 节点的LR信息需要开启内存检测功能才有有效输出 */
 LR[0]:0x0800414e
 LR[1]:0x08000cc2
 LR[2]:0x00000000

 pre node head LR info:   /* 通过LR信息，可以在汇编文件中查找前节点是哪里申请，然后排查其使用的准确性 */
 LR[0]:0x08004144
 LR[1]:0x08000cc2
 LR[2]:0x00000000
[ERR]Memory interity check error, cur node: 0x20003b10, pre node: 0x20003af0   /* 被破坏节点和其前节点的地址 */

Click to follow to learn about Huawei Cloud's fresh technology for the first time~