HUAWEI CLOUD GaussDB, with 5 core competencies, becomes the most beautiful one in SACC2021...

Abstract: Huawei Cloud NoSQL database architect Yu Wenlong was invited to participate in the 13th China System Architect Conference (SACC2021) and delivered an important speech, sharing the design concept and construction of GaussDB (for Redis) for the separation of storage and computing architecture The core competitiveness of its products, the use of innovative technology to build business agility, and drive the digital transformation of enterprises.

On May 20-22, the 13th China System Architects Conference (SACC2021) with the theme of "Digital Transformation and Architecture Reshaping" will be webcast live in the cloud. Huawei Cloud NoSQL database architect Yu Wenlong was invited to participate and delivered an important speech, sharing the GaussDB (for Redis) storage and computing separation architecture design concept and the core competitiveness of the product built, and building business agility with innovative technology to drive the enterprise Digital transformation.

As one of the most popular NoSQL databases in the industry, Redis has advantages in performance and data structure diversity. However, with the acceleration of the digitalization of enterprises, the disadvantages of open source Redis in terms of AOF expansion, snapshot performance, master-slave disconnection, high cost, and low capacity have become more and more obvious, which has further given birth to the technological innovation of the Redis ecosystem.

GaussDB (for Redis) is a cloud native database compatible with the Redis protocol independently developed by the Huawei Cloud database team. The database adopts a separate computing and storage architecture, breaking through the memory limitations of open source Redis, and can be easily expanded to PB-level storage.

Under the GaussDB (for Redis) computing and storage separation architecture model, the computing layer provides hot data caching, and the storage layer is responsible for placing all data on disk. Internally, they are interconnected through RDMA high-speed network, and algorithms are used to predict user access rules, realize automatic cold and heat exchange of data, and ultimately achieve the ultimate performance improvement.

The core part of the bottom layer of the architecture is based on Huawei's powerful and widely used self-developed distributed storage system DFV, which implements a cloud-native architecture of Share Everything, which fully utilizes the advantages of cloud-native elastic scaling and resource sharing, and perfectly avoid It solves the problems of open source Redis master-slave accumulation, master-slave inconsistency, fork jitter, memory utilization only 50%, big key blocking, gossip cluster management and other issues.

Based on this architecture, GaussDB (for Redis) has built five highly competitive capabilities, including strong consistency, high availability, cold and hot separation, elastic scaling, and high performance.

1. strongly consistent. Data replication is a matter of storage, so professional matters are left to a professional team. Through distributed storage DFV, GaussDB (for Redis) easily achieves 3 copies of strong consistency and can easily support 6 copies, which is the industry's first. Under the strong consistency architecture, users no longer have to worry about extreme data loss, inconsistency, OOM and other extreme problems caused by the master-slave accumulation of open source Redis, let alone business errors, such as inconsistencies in counters, current limiters, access statistics, hash fields, etc. .
2. highly available. open source Redis single shard and cluster, its data replication adopts a master-slave architecture, resulting in a cluster of N nodes, if a pair of master-slave (ie 2 nodes) is suspended at the same time, the entire cluster is unavailable. After GaussDB (for Redis) adopts the separation of storage and computing, each computing node can see and share all data. Therefore, for an instance of N nodes, up to N-1 node anomalies can be tolerated, and ultra-high availability is truly achieved.
3. cold and hot separation. order to give full play to the value of data, GaussDB (for Redis) implements separation of hot and cold data. Hot data is stored in memory and cold data is stored on disk. At the same time, LRU automatically eliminates cold data and real-time prediction algorithms load hot data. This not only makes the hot and cold data consistency strong, the interaction logic is simple, and it is business-friendly.
4. elastic scaling. data volume of 160af66a298167 expands to a certain scale, expansion is often considered a high-risk and difficult operation. GaussDB (for Redis) is based on a cloud-native architecture that decouples the expansion of computing resources and storage resources. Expansion of the computing layer does not require any data relocation, just modify the routing mapping, and it can be completed in seconds. The storage layer is a co-built super data lake with huge capacity and excellent dynamic capacity expansion. In addition, based on the 64MB fine-grained data partition, the entire expansion process is almost insensitive to the upper-level database business. Therefore, GaussDB (for Redis) can easily support the large-scale expansion of the business in the later stage, and truly achieve the on-demand expansion and purchase of computing/storage tiering.
5. high performance. benefits from multi-point reading and writing, and there is no spare node wasting resources. GaussDB (for Redis) is 2-3 times ahead of other cloud vendors in throughput and latency in the performance competition. At the same time, benefiting from the advantages of multi-threaded architecture, high compression ratio, and high memory utilization, GaussDB (for Redis) has better throughput and latency than open source Redis with the same memory.

The distributed architecture with separation of storage and computing is the general trend. At present, more and more companies have chosen to cloud their core business to this architecture, and the core capabilities of products built on this architecture have greatly promoted the development of corporate business and accelerated Enterprise digital transformation and upgrading. HUAWEI CLOUD is looking forward to having more enterprise customers in the future. Based on the core capabilities of GaussDB (for Redis), we will jointly promote the development of the industry and win new opportunities in this era!

Click to follow to learn about Huawei Cloud's fresh technology for the first time~