[Redis series] Redis learning fourteen, sorted_set preliminary research and sorting

What is sorted_set?

sorted_set is zset, which is one of the data in redis, an ordered set

The ordered set is a part of the set. The ordered set sets an extra score for each element, which is equivalent to an extra dimension. Redis also uses this dimension for sorting.

practical application

Connect redis-cli to redis-server, use help @sorted_set to view the commands supported by orderly combination

 # redis-cli -p 6379
127.0.0.1:6379> ping
PONG
127.0.0.1:6379>
127.0.0.1:6379> help @sorted_set

  BZPOPMAX key [key ...] timeout
  summary: Remove and return the member with the highest score from one or more sorted sets, or block until one is available
  since: 5.0.0
....

• summary

A summary of this command

• since

This command is available from which version of redis

for example

Add a key to sorted_set , this key has 3 members, and the branches corresponding to the 3 members are as follows:

member	Score
pig	9
dog	2
cat	6

 127.0.0.1:6379> zadd k1 9 pig 2 dog 6 cat
(integer) 3

Get all the values of the ordered set, the default is to display in an efficient to large way, because the data is stored in the redis memory, the result of the physical memory is from left to right, increasing one by one

 127.0.0.1:6379> ZRANGE k1 0 -1
1) "dog"
2) "cat"
3) "pig"

How to get the top 2 ranked from small to large?

 127.0.0.1:6379> ZRANGE k1 0 1 withscores
1) "dog"
2) "2"
3) "cat"
4) "6"

How about getting the top 2 rankings from largest to smallest?

The following is correct , use ZrevRANGE to get

 127.0.0.1:6379> ZrevRANGE k1 0 1 withscores
1) "pig"
2) "9"
3) "cat"
4) "6"

The following is wrong

 127.0.0.1:6379> ZRANGE k1 -2 -1 withscores
1) "cat"
2) "6"
3) "pig"
4) "9"

Example 2

Let's set scores for the following students and make a ranking according to the weight

k1	Fraction
xiaoming	90
zhangsan	40
lisi	60

k2	Fraction
xiaohong	30
zhangsan	70
wangwu	50

 127.0.0.1:6379> flushall
OK
127.0.0.1:6379> zadd k1 90 xiaoming 40 zhangsan 60 lisi
(integer) 3
127.0.0.1:6379> zadd k2 30 xiaohong 70 zhangsan 50 wangwu
(integer) 3
127.0.0.1:6379> ZUNIONSTORE unkey 2 k1 k2 weights 0.5 1
(integer) 5

Sort by weight, k1 accounts for 0.5, k2 accounts for 1, calculate the ranking , the actual example can be used to calculate the total score according to the weight

 127.0.0.1:6379> ZUNIONSTORE unkey 2 k1 k2 weights 0.5 1
(integer) 5
127.0.0.1:6379> Zrange unkey 0 -1 withscores
 1) "lisi"
 2) "30"
 3) "xiaohong"
 4) "30"
 5) "xiaoming"
 6) "45"
 7) "wangwu"
 8) "50"
 9) "zhangsan"
10) "90"

k1 and k1 take the maximum value of the members to rank , the actual example can be the highest score of multiple subjects to rank

 127.0.0.1:6379> ZUNIONSTORE unkey2 2 k1 k2 aggregate max
(integer) 5
127.0.0.1:6379> zrange unkey2 0 -1 withscores
 1) "xiaohong"
 2) "30"
 3) "wangwu"
 4) "50"
 5) "lisi"
 6) "60"
 7) "zhangsan"
 8) "70"
 9) "xiaoming"
10) "90"

So let's think about how the sorting of sorted_set is implemented?

sorted_set sorting implementation principle

Sorting is achieved by skiplist skip list , skiplist is a class balanced tree

skiplist is essentially a search structure, used to solve the search problem in the algorithm

Detailed Explanation of Redis Internal Data Structure As mentioned in this book, there are two types of solutions for finding problems:

• Based on various balanced trees
• Hash table based

The skiplist skip list does not belong to any of the above, it can be said to be a class balanced tree

Let's take an example:

For example, there are the following jump tables, with a total of 3 layers

Now insert the number 15 into this skip table

Use 15 to go to the first floor, it is bigger than 2, then go down

15 is smaller than 23 and larger than 2, then go down

15 is smaller than 23 and larger than 8, so 15 is inserted here

Insert here, the pointer of the third layer 8 points to 15, the pointer of 23 also points to 15

The second level 2 pointer points to 15, 15 points to 23

The pointer of the third level 2 also points to 15, and 15 points to NULL

According to the above example, we can understand that skiplist is a special linked list, called skip list, or skip list

We also found that with so many layers of linked lists, the lowest layer of linked list elements is the most complete, and other layers are sparse linked lists, and the pointers in these linked lists deliberately skip some nodes ( the higher-level linked list skips some nodes). more nodes )

This allows us to search for data in the high-level linked list first, then lower it layer by layer, and finally to the first layer of linked list to accurately determine the location of the data

In this way, many nodes are skipped, so it speeds up our search speed.

Whether it is adding, deleting, modifying or checking, you need to query first, first clearly find the position that needs to be operated, and then perform the operation.

Comparison of skiplist and balanced tree and hash table

	skiplist	Balanced tree	hash table
Algorithm implementation difficulty	Simple	difficult
Find a single key	The time complexity is O(log n)	The time complexity is O(log n)	The search time complexity is close to O(1) under the premise of keeping the hash value collision probability low higher performance
range lookup	Suitable for	Suitable for	Not suitable
Is range lookup complicated?	Very simple, just need to traverse the first level linked list for several steps after finding the small value.	Complex requires some modifications to the balanced tree
Insert and delete operations	Simple and fast, just need to modify the pointer of the adjacent node	May trigger adjustments to subtrees
memory usage	The average number of pointers contained in a flexible node is 1/(1-p) , depending on the size of the parameter p	Each node of the balanced tree contains 2 pointers (pointing to the left and right subtrees respectively)

We see that there is a structure definition for skiplist in redis src/server.h

 /* ZSETs use a specialized version of Skiplists */
typedef struct zskiplistNode {
    sds ele;
    double score;
    struct zskiplistNode *backward;
    struct zskiplistLevel {
        struct zskiplistNode *forward;
        unsigned long span;
    } level[];
} zskiplistNode;

typedef struct zskiplist {
    struct zskiplistNode *header, *tail;
    unsigned long length;
    int level;
} zskiplist;

typedef struct zset {
    dict *dict;
    zskiplist *zsl;
} zset;

zskiplist , skip list

length	skip table length
level	The current maximum level node of the skip table

zskiplist defines the real skiplist structure:

• Head pointer header and tail pointer tail .

• The length of the linked list length , that is, the total number of nodes contained in the linked list

  One thing to note here:

  The newly created skiplist contains an empty head pointer which is not included in the length count

• level represents the total number of layers of skiplist , which is the maximum number of layers of all nodes

zskiplistNode , the node of the skip list

score	Score
backward	back pointer
forward	forward pointer

zskiplistNode defines the node structure of skiplist :

• Stored are sds , zadd command is decoded before inserting data into skiplist , the purpose of this should be to facilitate searching lexicographically compare data
• score The field is the score corresponding to the data
• backward field is a pointer (forward pointer) to the previous node of the linked list, and each node has only one forward pointer, so only the first level linked list is a doubly linked list.

• level[] Store the pointer to the next node of the linked list of each layer (backward pointer)

  Each layer corresponds to a backward pointer, represented by the forward field.

• span value, each backward pointer also corresponds to a span , which indicates how many nodes the current pointer spans span used to calculate element ranking (rank)

Regarding the redis source code, the source code for creating nodes, inserting nodes, and deleting nodes is in src/t_zset.c . If you are interested in the detailed source code process, you can read it carefully, and it is still in the process of reading.

References:

• redis_doc
• reids source code src/t_zset.c and src/server.h

Welcome - like, follow, favorite

Friends, your support and encouragement are the motivation for me to persist in sharing and improve quality

Okay, here it is this time

Technology is open, and our mentality should be open. Embrace change, live in the sun, and move forward.

I'm the little devil boy Nezha , welcome to like, follow and collect, see you next time~