Will Redis data be deleted immediately when it expires?

Code brother, when the key reaches the expiration time, will Redis delete it immediately?

Let’s start with the conclusion. It will not be deleted immediately . Redis has two strategies for deleting expired data:

• Periodically select some data to delete;
• lazy delete;

This command is in Redis 2.4 version, the expiration time is not very precise, it may be between zero and one second.

As of Redis 2.6, the expiration error is 0 to 1 millisecond.

EXPIRE key seconds [ NX | XX | GT | LT] command can set the expiration time for the specified key. If the expiration time is not set, the key will always exist unless we explicitly delete it, such as executing the DEL command.

The so-called "cunning rabbits die, running dogs cook", kill them when they are useless, and "graduate" at the age of 35 is the same reason.

so panic...

EXPIRE Redis 版本7.0.0 开始： ---76c634741558370ae963033be8bbdcb6---添加了选项： NX 、 XX GT 、 LT options.

• NX: The expiration time is only set when the key has not expired;
• XX: Set the expiration time only when the key has expired;
• GT: Set the expiration time only when the new expiration time is greater than the current expiration time;
• LT: Set to expiry time only if the new expiry time is less than the current expiry time.

Expiration and Persistence

In the master-slave or cluster architecture, the clocks of the two machines are seriously out of sync. Is there any problem?

Key expiration information is represented by a Unix absolute timestamp .

In order for the expired operation to run normally, the time between machines must be stably synchronized, otherwise the expiration time will be inaccurate.

For example, two machines whose clocks are seriously out of sync undergo RDB transmission, and the slave time is set to 2000 seconds in the future. If a key of the master is set to survive for 1000 seconds, when the slave loads the RDB, the key will consider the key expired (because the slave The machine time is set to 2000 s in the future) and does not wait for 1000 s to expire.

lazy delete

Lazy deletion is very simple, that is, when there is a client request to query the key , check whether the key has expired, and if it expires, delete the key .

For example, when Redis receives the GET movie:小泽#玛……利亚.rmvb request from the client, it will first check whether key = movie:小泽#玛……利亚.rmvb has expired, and delete it if it expires.

The initiative to delete expired data is given to each access request.

This implementation is implemented by the expireIfNeeded function, the source code path: src/db.c .

 int expireIfNeeded(redisDb *db, robj *key, int force_delete_expired) {
      // key 没有过期，return 0
    if (!keyIsExpired(db,key)) return 0;
    if (server.masterhost != NULL) {
        if (server.current_client == server.master) return 0;
        if (!force_delete_expired) return 1;
    }

    if (checkClientPauseTimeoutAndReturnIfPaused()) return 1;

    /* Delete the key */
    deleteExpiredKeyAndPropagate(db,key);
    return 1;
}

delete regularly

It is definitely not enough to perform deletion only by judging whether the key has expired by client access, because some keys have expired, but no one will access them in the future. How to delete these data?

Don't let these data "occupy the ditch without shit".

The so-called periodic deletion, that is, Redis runs 10 times every 1 second by default (every 100 ms), randomly extracts some keys with an expiration time set each time, checks whether they expire, and deletes them directly if they are expired.

Note: Instead of checking all libraries, all keys in one run, a random number of keys are checked.

Specific steps are as follows:

1. Randomly select 20 keys from all key sets with expiration time set;
2. Delete all expired key data found in "Step 1";
3. "Step 2" ends, and if the expired key exceeds 25%, continue to "Step 1".

Removed source code expire.c 的 activeExpireCycle 函数实现 .

This means that the maximum number of expired keys at any one time is equal to the maximum number of write operations per second divided by 4.

Why not check all keys with expiration time set?

Think about it, assuming that Redis stores 100w keys, all have an expiration time set, and 100w keys are checked every 100 milliseconds. The CPU is all wasted on checking expired keys, and Redis is useless.

Note: whether it is timed deletion or lazy deletion. When the data is deleted , master will generate the deleted instruction records to the AOF and slave nodes .

Brother code, if there is too much expired data, it cannot be completely deleted by regular deletion (more than 25% of expired keys are still deleted after each deletion), and these keys will never be requested by the client again, that is, they cannot be deleted in a lazy way. how?

Will it cause Redis to run out of memory and how to break it?

This question is a good question, and the answer is to take the memory elimination mechanism .

That's all for today. If you talk too much, everyone will easily choke to death in the massive amount of knowledge. Life-saving is the most important thing. As for the details of the memory elimination mechanism, please see the next breakdown.

References