Redis core principle and practice-source code analysis of Redis startup process

The Redis server is responsible for receiving and processing user requests and providing services for users.
The format of the startup command for the Redis server is as follows:

redis-server [ configfile ] [ options ]

The configfile parameter specifies the configuration file. The options parameter specifies the startup configuration items, which can override the configuration items in the configuration file, such as

redis-server /path/to/redis.conf --port 7777 --protected-mode no

This command starts the Redis service, specifies the configuration file /path/to/redis.conf, and gives two startup configuration items: port and protected-mode.

This article analyzes the Redis startup process by reading the Redis source code, and the content is taken from the new book "Redis Core Principles and Practices".
This article involves many concepts of Redis, such as event looper, ACL, Module, LUA, slow log, etc. These functions are analyzed in detail in the author's new book "Redis Core Principles and Practice". Interested readers can refer to this book.

Server definition

Tip: If there is no special description, the codes in this chapter are in server.h and server.c.

The server.h/redisServer structure is defined in Redis, which stores Redis server information, including server configuration items and runtime data (such as network connection information, database redisDb, command table, client information, slave server information, statistical information, etc.) .

struct redisServer {
    pid_t pid;                  
    pthread_t main_thread_id;         
    char *configfile;           
    char *executable;           
    char **exec_argv;    
    ...
}

There are many attributes in redisServer, so I won’t list them one by one here. We will explain the relevant server attributes when we analyze the specific functions.
A redisServer global variable is defined in server.h:

extern struct redisServer server;

The server variables mentioned in this book, unless otherwise specified, refer to the redisServer global variables. For example, part 1 said that attributes such as server.list_max_ziplist_size refer to the attributes of the variable.
You can use the INFO command to obtain server information. The command mainly returns the following information:

• server: General information about the Redis server.
• clients: client connection information.
• memory: memory consumption related information.
• persistence: RDB and AOF persistence information.
• stats: General statistics.
• replication: master/replica replication information.
• cpu: CPU consumption information.
• commandstats: Redis command statistics.
• cluster: Redis Cluster cluster information.
• modules: Modules module information.
• keyspace: Statistics related to the database.
• errorstats: Redis error statistics.

In addition to statistical data such as memory and cpu in the response content of the INFO command, most of the other data are stored in redisServer.

main function

The server.c/main function is responsible for starting the Redis service:

int main(int argc, char **argv) {
    ...
    // [1]
    server.sentinel_mode = checkForSentinelMode(argc,argv);
    // [2]
    initServerConfig();
    ACLInit(); 
    
    moduleInitModulesSystem();
    tlsInit();

    // [3]
    server.executable = getAbsolutePath(argv[0]);
    server.exec_argv = zmalloc(sizeof(char*)*(argc+1));
    server.exec_argv[argc] = NULL;
    for (j = 0; j < argc; j++) server.exec_argv[j] = zstrdup(argv[j]);

    // [4]
    if (server.sentinel_mode) {
        initSentinelConfig();
        initSentinel();
    }

    // [5]
    if (strstr(argv[0],"redis-check-rdb") != NULL)
        redis_check_rdb_main(argc,argv,NULL);
    else if (strstr(argv[0],"redis-check-aof") != NULL)
        redis_check_aof_main(argc,argv);

    // more
}

[1] Check whether the Redis server is started in sentinel mode.
[2] The initServerConfig function initializes the properties of the configuration items recorded in redisServer to default values. The ACLInit function initializes the ACL mechanism, and the moduleInitModulesSystem function initializes the Module mechanism.
[3] Record the executable path and startup parameters of the Redis program for subsequent restart of the server.
[4] If starting in Sentinel mode, initialize the Sentinel mechanism.
[5] If the startup program is redis-check-rdb or redis-check-aof, execute the redis_check_rdb_main or redis_check_aof_main function, they try to check and repair the RDB, AOF file and then exit the program.
After Redis is compiled, 5 executable programs will be generated:

• redis-server: Redis execution program.
• redis-sentinel: Redis Sentinel execution program.
• redis-cli: Redis client program.
• redis-benchmark: Redis performance stress testing tool.
• redis-check-aof, redis-check-rdb: tools for checking and repairing RDB and AOF persistent files.

Continue to analyze the main function:

int main(int argc, char **argv) {
    ...
    if (argc >= 2) {
        j = 1; 
        sds options = sdsempty();
        char *configfile = NULL;

        // [6]
        if (strcmp(argv[1], "-v") == 0 ||
            strcmp(argv[1], "--version") == 0) version();
        ...

        // [7]
        if (argv[j][0] != '-' || argv[j][1] != '-') {
            configfile = argv[j];
            server.configfile = getAbsolutePath(configfile);
            zfree(server.exec_argv[j]);
            server.exec_argv[j] = zstrdup(server.configfile);
            j++;
        }

       // [8]
        while(j != argc) {
            ...
        }
        // [9]
        if (server.sentinel_mode && configfile && *configfile == '-') {
            ...
            exit(1);
        }
        // [10]
        resetServerSaveParams();
        loadServerConfig(configfile,options);
        sdsfree(options);
    }
    ...
}

[6] Give priority to commands such as -v, --version, --help, -h, and --test-memory.
The strcmp function compares two strings str1 and str2. If str1=str2, it returns zero; if str1<str2, it returns a negative number; if str1>str2, it returns a positive number.
[7] If the second parameter of the startup command does not start with "--", it is a configuration file parameter. The configuration file path is converted to an absolute path and stored in server.configfile.
[8] Read the startup configuration items in the startup command and splice them into a string.
[9] To start in Sentinel mode, the configuration file must be specified, otherwise it will directly report an error and exit.
[10] The config.c/resetServerSaveParams function resets the server.saveparams property (this property stores the RDB SAVE configuration). The config.c/loadServerConfig function loads all configuration items from the configuration file, and uses the startup command configuration items to overwrite the configuration items in the configuration file.

Tip: The configs array in config.c defines the correspondence between most configuration options and server attributes:

standardConfig configs[] = {
    createBoolConfig("rdbchecksum", NULL, IMMUTABLE_CONFIG, server.rdb_checksum, 1, NULL, NULL),
    createBoolConfig("daemonize", NULL, IMMUTABLE_CONFIG, server.daemonize, 0, NULL, NULL),
    ...
}

The configuration item rdbchecksum corresponds to the server.rdb_checksum property, and the default value is 1 (that is, the bool value is yes), and other configuration items are analogous to this. If the reader needs to find the server attribute and default value corresponding to the configuration item, you can find it.

Continue to analyze the main function below:

int main(int argc, char **argv) {
    ...
    // [11]    
    server.supervised = redisIsSupervised(server.supervised_mode);
    int background = server.daemonize && !server.supervised;
    if (background) daemonize();
    // [12]
    serverLog(LL_WARNING, "oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo");
    ...
    
    // [13]
    initServer();
    if (background || server.pidfile) createPidFile();
    ...

    if (!server.sentinel_mode) {
        ...
        // [14]
        moduleLoadFromQueue();
        ACLLoadUsersAtStartup();
        InitServerLast();
        loadDataFromDisk();
        if (server.cluster_enabled) {
            if (verifyClusterConfigWithData() == C_ERR) {
                ...
                exit(1);
            }
        }
        ...
    } else {
        // [15]
        InitServerLast();
        sentinelIsRunning();
        ...
    }

    ...
    // [16]
    redisSetCpuAffinity(server.server_cpulist);
    setOOMScoreAdj(-1);
    // [17]
    aeMain(server.el);
    // [18]
    aeDeleteEventLoop(server.el);
    return 0;
}

[11] The server.supervised attribute specifies whether to start Redis with the upstart service or the systemd service. If server.daemonize is configured and server.supervised is not configured, Redis will be started as a daemon.
[12] Print the startup log.
[13] The initServer function initializes Redis runtime data, and the createPidFile function creates a pid file.
[14] If the non-Sentinel mode is started, complete the following operations:
(1) The moduleLoadFromQueue function loads the Module module specified by the configuration file;
(2) ACLLoadUsersAtStartup function loads ACL user control list;
(3) The InitServerLast function is responsible for creating background threads and I/O threads. This step needs to be executed after the Module module is loaded;
(4) The loadDataFromDisk function loads AOF or RDB files from the disk.
(5) If you start in Cluster mode, you also need to verify whether the loaded data is correct.
[15] If it is started in Sentinel mode, call the sentinelIsRunning function to start the Sentinel mechanism.
[16] As much as possible, bind the Redis main thread to the CPU list configured by server.server_cpulist. Redis 4 starts to use multithreading. This operation can reduce unnecessary thread switching and improve performance.
[17] Start the event looper. The event looper is an important component in Redis. During Redis operation, the service is provided by the event looper.
[18] Execution to this point indicates that the Redis service has stopped, the aeDeleteEventLoop function clears the event in the event looper, and finally exits the program.

Redis initialization process

Let's take a look at the initServer function, which is responsible for initializing Redis runtime data:

void initServer(void) {
    int j;
    // [1]
    signal(SIGHUP, SIG_IGN);
    signal(SIGPIPE, SIG_IGN);
    setupSignalHandlers();
    // [2]
    makeThreadKillable();
    // [3]
    if (server.syslog_enabled) {
        openlog(server.syslog_ident, LOG_PID | LOG_NDELAY | LOG_NOWAIT,
            server.syslog_facility);
    }

    // [4]
    server.aof_state = server.aof_enabled ? AOF_ON : AOF_OFF;
    server.hz = server.config_hz;
    server.pid = getpid();
    ...

    
    // [5]
    createSharedObjects();
    adjustOpenFilesLimit();
    // [6]
    server.el = aeCreateEventLoop(server.maxclients+CONFIG_FDSET_INCR);
    if (server.el == NULL) {
        ...
        exit(1);
    }
    
    // more
}

[1] Set the UNIX signal processing function so that the Redis server exits the program after receiving the SIGINT signal.
[2] Set the thread to respond to the CANCEL signal at any time to terminate the thread in order to stop the program.
[3] If the Unix system log is enabled, call the openlog function to establish an output connection with the Unix system log to output the system log.
[4] Initialize the relevant attributes responsible for storing runtime data in the server.
[5] The createSharedObjects function creates a shared data set. These data can be shared and used in various scenarios, such as small numbers 0～9999, commonly used strings +OK\r\n (command processing success response string), +PONG\r\ n (ping command response string). The adjustOpenFilesLimit function attempts to modify environment variables to increase the upper limit of file descriptors that the system allows to open, and avoid errors caused by a large number of client connections (Socket file descriptors).
[6] Create an event looper.
UNIX programming: Signals are also called soft interrupts. Signals are a way to handle asynchronous events provided by UNIX. The program tells the system kernel what to do after the signal is generated by setting a callback function. Many scenes in the system will generate signals, such as:

• The user presses certain terminal keys to cause the terminal to generate a signal. For example, if the user presses the interrupt key (usually Ctrl+C key combination) on the terminal, the SIGINT signal will be sent to notify the program to stop running.
• Some specific events have occurred in the system. For example, when the timer set by the alarm function expires, the kernel sends a SIGALRM signal, or when a process terminates, the kernel sends a SIGCLD signal to its parent process.
• Certain hardware exceptions, for example, a divisor of 0, invalid memory references.
• The program uses functions to send signals, for example, calling the kill function to send any signal to another process.
  Interested readers can learn more about UNIX programming related content on their own.

Then analyze the initServer function:

void initServer(void) {    
    server.db = zmalloc(sizeof(redisDb)*server.dbnum);

    // [7]
    if (server.port != 0 &&
        listenToPort(server.port,server.ipfd,&server.ipfd_count) == C_ERR)
        exit(1);
    ...

    // [8]
    for (j = 0; j < server.dbnum; j++) {
        server.db[j].dict = dictCreate(&dbDictType,NULL);
        server.db[j].expires = dictCreate(&keyptrDictType,NULL);
        ...
    }

    // [9]
    evictionPoolAlloc(); 
    server.pubsub_channels = dictCreate(&keylistDictType,NULL);
    server.pubsub_patterns = listCreate();
    ...
}

[7] If the server.port is configured, the TCP Socket service is turned on to receive user requests. If server.tls_port is configured, then the TLS Socket service is enabled, and Redis 6.0 starts to support TLS connections. If server.unixsocket is configured, open UNIX Socket service. If none of the above 3 options are configured, an error will be reported and exit.
[8] Initialize the database server.db, which is used to store data.
[9] The evictionPoolAlloc function initializes the LRU/LFU sample pool, which is used to implement the LRU/LFU approximation algorithm.
Continue to initialize the relevant attributes of the runtime data stored in the server:

void initServer(void) {
    ...
    // [10]
    if (aeCreateTimeEvent(server.el, 1, serverCron, NULL, NULL) == AE_ERR) {
        serverPanic("Can't create event loop timers.");
        exit(1);
    }

    // [11]
    for (j = 0; j < server.ipfd_count; j++) {
        if (aeCreateFileEvent(server.el, server.ipfd[j], AE_READABLE,
            acceptTcpHandler,NULL) == AE_ERR)
            {
                serverPanic(
                    "Unrecoverable error creating server.ipfd file event.");
            }
    }
    ...

    // [12]
    aeSetBeforeSleepProc(server.el,beforeSleep);
    aeSetAfterSleepProc(server.el,afterSleep);

    // [13]
    if (server.aof_state == AOF_ON) {
        server.aof_fd = open(server.aof_filename,
                               O_WRONLY|O_APPEND|O_CREAT,0644);
        ...
    }

    // [14]
    if (server.arch_bits == 32 && server.maxmemory == 0) {
        ...
        server.maxmemory = 3072LL*(1024*1024); /* 3 GB */
        server.maxmemory_policy = MAXMEMORY_NO_EVICTION;
    }
    // [15]
    if (server.cluster_enabled) clusterInit();
    replicationScriptCacheInit();
    scriptingInit(1);
    slowlogInit();
    latencyMonitorInit();
}

[10] Create a time event, the execution function is serverCron, which is responsible for processing timed tasks in Redis, such as cleaning up expired data, generating RDB files, etc.
[11] Register and listen for file events of type AE_READABLE for TCP Socket, TSL Socks, and UNIX Socket. The event processing functions are acceptTcpHandler, acceptTLSHandler, and acceptUnixHandler. These functions are responsible for receiving new connections in the Socket. The book will later analyze the acceptTcpHandler function in detail. .
[12] Register the hook function of the event looper, and the event looper will call the hook function before and after each block.
[13] If AOF is turned on, open the AOF file in advance.
[14] If Redis runs on a 32-bit operating system, since the memory space of the 32-bit operating system is limited to 4GB, the memory used by Redis is limited to 3GB to avoid Redis server crash due to insufficient memory.
[15] If it is started in Cluster mode, call the clusterInit function to initialize the Cluster mechanism.

• The replicationScriptCacheInit function initializes the server.repl_scriptcache_dict property.
• The scriptingInit function initializes the LUA mechanism.
• The slowlogInit function initializes the slow log mechanism.
• The latencyMonitorInit function initializes the latency monitoring mechanism.

Summarize:

• The redisServer structure stores server configuration items and runtime data.
• server.c/main is the Redis startup method, responsible for loading the configuration, initializing the database, starting the network service, creating and starting the event looper.

At the end of the article, I will introduce the new book "Redis Core Principles and Practice". This book summarizes the design and implementation of Redis core functions through in-depth analysis of Redis 6.0 source code. By reading this book, readers can deeply understand Redis's internal mechanisms and latest features, and learn Redis-related data structures and algorithms, Unix programming, storage system design, distributed system architecture, and a series of knowledge.
With the consent of the editor of the book, I will continue to publish some chapters in the personal technical public account (binecy) as a preview of the book. You are welcome to check it out, thank you.

platform preview: 1617a437e31969 "Redis Core Principles and Practice"
Jingdong link