Faster than Elasticsearch! RediSearch + RedisJSON = King Fried!

Redis is a very commonly used in-memory data storage middleware in development. It was basically used for in-memory storage before. Recently, I found that Redis has launched many enhanced modules. For example, RedisJSON can support the storage of native JSON objects, use RediSearch as a search engine, and support Chinese search! Today, I will bring you the practice of using RediSearch+RedisJSON as a search engine, and I hope it will be helpful to you!

SpringBoot actual e-commerce project mall (50k+star) address: https://github.com/macrozheng/mall

Introduction to RedisMod

First, let's introduce RedisMod, which is a series of enhanced modules for Redis. With the support of RedisMod, the function of Redis will become very powerful. Currently RedisMod includes the following enhanced modules:

• RediSearch: a full-featured search engine;
• RedisJSON: native support for JSON type;
• RedisTimeSeries: time series database support;
• RedisGraph: graph database support;
• RedisBloom: native support for probabilistic data;
• RedisGears: programmable data processing;
• RedisAI: Real-time model management and deployment for machine learning.

Install

First we need to install Redis with all RedisMods, it is very convenient to use Docker to install!

• Use the following command to download the image of RedisMod;

docker pull redislabs/redismod:preview

• Run the RedisMod service in a container.

docker run -p 6379:6379 --name redismod \
-v /mydata/redismod/data:/data \
-d redislabs/redismod:preview

RedisJSON

With the RedisJSON module, Redis can store native JSON type data, through which you can easily access various attributes in JSON, similar to MongoDB, let's experience it, here we will use [RedisInsight]( ) to operate Redis.

• First, use the JSON.SET command to add JSON type key-value pairs and several commodity object data to Redis. Since JSON is a tree structure, the $ symbol is used to represent adding data to the root node of JSON;

JSON.SET product:1 $ '{"id":1,"productSn":"7437788","name":"小米8","subTitle":"全面屏游戏智能手机 6GB+64GB 黑色 全网通4G 双卡双待","brandName":"小米","price":2699,"count":1}'
JSON.SET product:2 $ '{"id":2,"productSn":"7437789","name":"红米5A","subTitle":"全网通版 3GB+32GB 香槟金 移动联通电信4G手机 双卡双待","brandName":"小米","price":649,"count":5}'
JSON.SET product:3 $ '{"id":3,"productSn":"7437799","name":"Apple iPhone 8 Plus","subTitle":"64GB 红色特别版 移动联通电信4G手机","brandName":"苹果","price":5499,"count":10}'

• After the data is inserted successfully, you will see the following information in RedisInsight. JSON data supports formatting and highlighting;

• Next, you can obtain the value of the JSON type key-value pair through the JSON.GET command;

JSON.GET product:1

• You can also get only the specified property of the value. In RedisJSON, you need to start with . when getting the property in the JSON object;

JSON.GET product:1 .name .subTitle

• You can also get the JSON object type through the JSON.TYPE command.

JSON.TYPE product:1 .

RediSearch

Through the RediSearch module, Redis can become a powerful full-text search engine, and natively supports Chinese search, let's experience it below!

• Before using RediSearch to search data, we have to create an index first. The syntax of index building is a bit complicated, let's take a look first;

FT.CREATE {index}
  [ON {data_type}]
     [PREFIX {count} {prefix} [{prefix} ..]
     [LANGUAGE {default_lang}]
  SCHEMA {identifier} [AS {attribute}]
      [TEXT | NUMERIC | GEO | TAG ] [CASESENSITIVE]
      [SORTABLE] [NOINDEX]] ...

• Use the FT.CREATE command to create an index. The meaning of the parameters in the syntax is as follows;

  • index: index name;
  • data_type: The data type for indexing, currently supports JSON or HASH;
  • PREFIX: Through it, you can select the data prefix that needs to be indexed. For example, PREFIX 1 "product:" indicates that the data prefixed with product: in the key is indexed;
  • LANGUAGE: Specifies the default language of the TEXT type attribute, which can be set to Chinese using Chinese;
  • identifier: specifies the attribute name;
  • attribute: specifies the attribute alias;
  • TEXT | NUMERIC | GEO | TAG: These are all optional types of attributes;
  • SORTABLE: The specified property can be sorted.
• After reading the grammar, it may not be easy to understand. Just try to build an index on the previous product data and you will understand;

FT.CREATE productIdx ON JSON PREFIX 1 "product:" LANGUAGE chinese SCHEMA $.id AS id NUMERIC $.name AS name TEXT $.subTitle AS subTitle TEXT $.price AS price NUMERIC SORTABLE $.brandName AS brandName TAG

• After the index is established, we can use FT.SEARCH to view the data, for example, use * to query all;

FT.SEARCH productIdx *

• Since we set the price field to SORTABLE , we can return the product information in descending order of price ;

FT.SEARCH productIdx * SORTBY price DESC

• You can also specify the returned fields;

FT.SEARCH productIdx * RETURN 3 name subTitle price

• We set brandName as TAG type, we can use the following statement to query the products whose brand is Xiaomi or Apple;

FT.SEARCH productIdx '@brandName:{小米 | 苹果}'

• Since price is of type NUMERIC , we can use the following statement to query products whose price is 500~1000 ;

FT.SEARCH productIdx '@price:[500 1000]'

• Fuzzy query can also be performed by prefix, similar to LIKE in SQL, represented by * ;

FT.SEARCH productIdx '@name:小米*'

• Directly specify search keywords in FT.SEARCH , you can perform a global search for all attributes of type TEXT , and support Chinese search, for example, we search for products containing the black field of ;

FT.SEARCH productIdx '黑色'

• Of course, we can also specify the search field, such as searching for products with a red field of in the subtitle;

FT.SEARCH productIdx '@subTitle:红色'

• The index can be deleted through the FT.DROPINDEX command. If the DD option is added, the data will be deleted together;

FT.DROPINDEX productIdx

• The index status can be viewed through the FT.INFO command;

FT.INFO productIdx

• The search syntax of RediSearch is more complicated, but we can use it by comparing with SQL. For details, please refer to the following table.

Compare Elasticsearch

Redis officially announced the performance comparison test of RediSearch and Elasticsearch, you can take a look.

indexing ability

Indexing 5.6 million (5.3GB) documents from Wikipedia, RediSearch took 221s , Elasticsearch took 349s , and RediSearch was 58% faster!

query capability

After the data is indexed, 32 clients are used to retrieve two words. The throughput of RediSearch reaches 12.5K ops/sec , and the throughput of Elasticsearch is 3.1K ops/sec . RediSearch is 4 times faster than Elasticsearch. At the same time, the latency of RediSearch is 8ms , while that of Elasticsearch is 10ms , and the latency of RediSearch is slightly lower!

Summarize

After so many years of development, Redis has become more and more powerful. It is not only a caching tool, but more like a database. RediSearch gives us another option to implement the search function, and the performance is also very good. If you are doing search-related functions, you can consider it!

If you want to learn more Redis combat skills, you can try this practical project with a full set of tutorials (50K+Star): https://github.com/macrozheng/mall

References

• Official documentation: https://developer.redis.com/howtos/redisjson/
• Reference Manual: https://oss.redis.com/redisearch/
• Performance test: https://redis.com/blog/search-benchmarking-redisearch-vs-elasticsearch/