Thinking behind the trinity strategy of cloud native gateway open source, self-research, and commercialization

*Author: Rufen

Alibaba's trinity strategy interpretation of cloud native gateways is open source, self-developed, and commercialized. At present, cloud native gateways have been officially commercialized, aiming to provide users with a more reliable, lower cost, and higher efficiency enterprise-level that conforms to the K8s Ingress standard Gateway products, more details will be explained in detail in the live broadcast room at 15:00 pm on October 26, details stamp: https://yqh.aliyun.com/live/detail/26605

Interpretation of the overall strategy of Alibaba Cloud's native trinity

At present, open source software has become the main platform for enterprise software innovation and development. In this context, Alibaba Cloud has proposed a trinity strategy, namely open source, self-research and commercialization. Research will further polish the performance and high availability of the software, and then open it to all users in the form of commercial products. At the same time, it will continue to contribute to the open source community, and finally form a three-in-one rotating flywheel. The overall strategy is as follows:

The cloud native team business is divided into three layers as a whole: BaaS, Runtime, and Serverless. Each layer uses open source software as the core, and integrates with Ali's business to expand internally in open source core integrators, and then commercialize it after large-scale production verification.

Advantages of Alibaba Cloud Trinity

In the trinity, open source, self-research and commercialization all have their own focus or characteristics. Let's take a look at the illustration:

Relying on open source as the core, Alibaba’s large and complex business scenarios are required to drive self-research and expansion. After large-scale production-level verification, it is promoted to commercialization. At the same time, it submits self-research functions suitable for popularization to the open source community. This is Ali The advantage of the cloud trinity lies.

Cloud native gateway development track

The birth background of cloud native gateway

The initial creation of the cloud native gateway came from the internal "local life battle", which began at the "Alipay 2020 Partner Conference", at which Alipay announced its upgrade to an open platform for digital life. See this link for details. The core technical goal of this battle is to achieve direct RPC calls between the Alibaba business domain and the Ant business domain. Because Alibaba and the Ant business domain network are isolated, that is, the network is not connected, it is natural to think of using a gateway to solve this problem. A brief introduction of the "Battle of Local Life" is as follows:

Technical Selection of Cloud Native Gateway

To use the gateway to solve the problem of RPC interoperability between Alibaba and Ant across business domains, we must first select the technology of the gateway. I believe everyone is more or less aware of Alibaba’s open source reverse proxy program Tengine. Tengine has unified access to the gateway within Alibaba. AServer was used. Our team was responsible for the development and operation of AServer at that time. At the same time, our team was also responsible for the implementation of Alibaba Service Mesh. Whether it is Tengine or Istio + Envoy, they are familiar with the architecture. Although some other software has been investigated during the selection, considering the high requirements of the gateway for performance and reliability, combined with our own gateway operation and maintenance experience, we decided to see whether Tengine and Envoy can meet our business needs. During the comparison, we listed four key points. The comparison is as follows:

Here is a mention of "Why do we think that the hot update of the configuration is very important?" The configuration update of Tengine/Nginx requires reload, which requires restarting the worker process, which will cause traffic jitter when restarting, and the impact on long connections is particularly obvious. When the cluster size of the gateway is very large, it is not possible to reload at will. At this time, it will cause a contradiction: after the business submits the configuration to the gateway, it hopes to verify quickly, and is limited by the reload mechanism. The gateway cannot satisfy the business because of its stability. The appeal of quick verification and quick trial and error. How to solve this? One is to use a two-tier gateway, that is, a traffic gateway + a service gateway, and the other is that the gateway natively supports hot configuration updates.

After comparing the preliminary selection of Envoy, we further investigated the trend of Envoy as a gateway in the industry. The conclusion is that Envoy is currently the fastest growing program in K8s as the Ingress Provider (Ingress Provider refers to the specific implementation of the K8s Ingress specification, because of the K8s Ingress itself It's just the specification definition, which is the specification definition of the gateway for external traffic to enter the cluster under K8s), see the following figure:

The development history of cloud native gateways

From the initial community of Istio + Envoy, to the self-research and expansion of Alibaba, to the large-scale generation and verification, the cloud native gateway finally completes the release of commercial products. The whole process is described as follows:

The formation of the cloud native gateway trinity flywheel

After the official commercialization of the cloud-native gateway, the cloud-native gateway has completed the complete development of open source, self-research, and commercialization, and the three-in-one rotating flywheel has taken shape. We will also continue to contribute our strength to the open source community, such as submitting Dubbo support to the Envoy community, cloud native messaging team submitting RocketMQ support, and other small issues.

After understanding the development of the cloud-native gateway trinity, next I will specifically introduce the product positioning and advantages of the cloud-native gateway.

Introduction to Cloud Native Gateway

Traditional gateway classification and deployment mode

In the industry, gateways are usually divided into two categories: traffic gateways and business gateways. Traffic gateways mainly provide global policy configuration that has nothing to do with back-end services. For example, Ali’s internal unified access gateway Tengine is a typical traffic gateway. ; Service gateway, as the name suggests, mainly provides independent service domain level, tightly coupled with back-end business strategy configuration, as the application architecture model evolves from a single to the current distributed microservices, the service gateway also has a new name-microservice gateway ( The illustration is as follows). In the current cloud-native era dominated by container technology and K8s, is the next-generation gateway model still the same?

Product portrait of next-generation gateway

Just like the question in the above figure: In the cloud-native era dominated by container technology and K8s, will the next-generation gateway model still be the traditional two-tier architecture of traffic gateway and microservice gateway? With this problem and combined with the gateway technology and operation and maintenance experience accumulated in Alibaba, we tried to make a product portrait for the next-generation gateway products, as follows:

As a next-generation gateway product, we will explain some of the core elements:

• Cloud native: To support the standard K8s Ingress, K8s Gateway API, and K8s service discovery, K8s has become a cloud OS in the cloud native era, and the network inside and outside the K8s native cluster is isolated. The specification for external traffic entering the K8s cluster is defined as K8s Ingress, K8s Gateway API is a further evolution of K8s Ingress, based on this, it is necessary to support this feature as a next-generation gateway.
• Embrace open source: We must build a gateway based on the open source ecosystem and use open source to help open source. I believe this should be familiar to everyone.
• High scalability: The capabilities of any gateway cannot cover all user demands. It must have scalability. For example, the booming development of K8s and its open scalability are indispensable.
• Service governance: As the application architecture evolves to distributed microservices, the gateway itself provides traffic scheduling capabilities for back-end services, and its ability to support basic service governance is natural.
• Abundant observability: While the distributed microservice architecture brings benefits such as improved collaboration efficiency, it also brings greater challenges to troubleshooting and operation and maintenance. As a traffic bridgehead, the gateway needs to have abundant observable data to help users To locate the problem.

Positioning of Cloud Native Gateway

Based on the above-mentioned understanding of the next-generation gateway, we took the lead in launching a cloud-native gateway inside Alibaba. It was successfully deployed in multiple services and passed the test of the Double 11 promotion. The cloud-native gateway is illustrated as follows:

Product advantages of cloud native gateways

More economical: Combining the traffic gateway and the microservice gateway into one, the user resource cost is reduced by 50%

Under the microservice architecture in the virtualization era, the business usually adopts a two-tier architecture of traffic gateway + microservice gateway. The traffic gateway is responsible for north-south traffic scheduling and security protection, and the microservice gateway is responsible for east-west traffic scheduling and service governance. In the cloud-native era dominated by K8s, Ingress has become the gateway standard for the K8s ecosystem, giving the gateway a new mission, making it possible to combine the traffic gateway + microservice gateway into one.

The cloud-native gateway released by Alibaba Cloud MSE this time changes the two-layer gateway into one layer without any discount on the capabilities, which not only saves 50% of resource costs, but also reduces operation and maintenance and usage costs. The schematic diagram of the deployment structure is as follows, the left is the traditional gateway mode, and the right is the next-generation cloud native gateway mode.

In the context of microservices, rich observable capabilities are also the basic core demands of users. Based on this, the cloud native gateway integrates the Alibaba Cloud application real-time monitoring service ARMS by default, providing rich observable data, and this function is free for users.

More secure: Provides a wealth of authentication and authentication capabilities to reduce the cost of secure access for customers

Authentication authentication is the rigid demand of customers for the gateway. The MSE cloud native gateway not only provides regular JWT authentication, but also provides OIDC authentication based on the authorized open network standard OAuth 2.0. At the same time, the MSE cloud native gateway naturally supports Alibaba Cloud’s application identity service IDaaS to help customers achieve three-party authentication logins for Alipay, Taobao, Tmall, etc., and supports extensions of authentication and authentication functions in the form of plug-ins to reduce customers’ secure access. Cost. The existing authentication functions are as follows:

More unified: the gateway is directly connected to the back-end service, opens up multiple service sources of Nacos/Eureka/K8s, and is the first to support the Apache Dubbo3.0 protocol

Open source has become one of the driving forces to promote software development, and community-oriented, open commercial products have more vitality.

Envoy is one of the most popular Ingress implementations in the K8s community, and it is becoming the standard technical solution for traffic portals in the cloud-native era. The MSE cloud native gateway is built on Envoy and Istio to achieve a unified control plane control, and is directly connected to back-end services, supports Dubbo3.0 and Nacos, opens up Alibaba Cloud Container Service ACK, and automatically synchronizes service registration information. MSE Cloud Native Gateway's support for Dubbo 3.0 and Nacos has been launched in Dingding business first. The following figure shows the deployment diagram of Dingding Dubbo 3.0 as follows:

More stable: The technology has been accumulated for a long time, and it has passed the test of 2020 Double 11, carrying hundreds of thousands of requests per second

Commercial products are not overnight.

The MSE cloud native gateway has already experienced thousands of trials within Alibaba. It has been used in Ali’s business systems such as Alipay, Dingding, Taobao, Tmall, Youku, Fliggy, Word of Mouth, etc., and has passed the test of the massive number of requests on the 2020 Double 11, and the big promotion day can easily carry hundreds of thousands of requests per second. , The daily request volume has reached the level of tens of billions.

Cloud Native Gateway Applicable Scenarios

The cloud native gateway can currently cover all business scenarios in north-south and east-west directions, that is, it can support traditional registration centers such as Nacos, it can also support K8s Service, and it can also support traditional ECS. The following diagram illustrates the following:

Nginx gateway migration cloud native gateway practice case

Youku Nginx Gateway Migration Case

There are three gateways built with Nginx in Youku's business, and some business expansions have been made using Lua scripts. However, with the iteration of personnel, the operation and maintenance of the gateway has become more and more difficult, mainly including the difficulty of maintaining Lua scripts, and the fact that configuration changes are not real-time. There are more and more conflicts between gateway operation and maintenance, configuration change reload and rapid business iteration. Therefore, we cooperated with the business to complete the smooth migration of the Nginx gateway to the cloud native gateway. In order to ensure smooth migration and low cost, specialize in the cloud native gateway. Expansion has been made for Nginx's error page and other functions. As shown below:

After seeing the picture above, readers may ask why a two-tier gateway structure is used? That is, AServer (traffic gateway) + business self-built gateway. This is because for Alibaba’s super-large traffic business, a two-tier architecture is adopted. The traffic gateway is responsible for security protection, and the cost of global traffic scheduling will be lower. Business BUs do repetitive things, and the self-built gateway of the business can meet the requirements of fast iteration of the business itself after hanging on the traffic gateway; but for non-ultra-large-scale businesses, the use of a two-tier architecture will increase the complexity of operation and maintenance.

Ingress-nginx migration plan

Although the growth of users using the Envoy architecture ranks first in K8s Ingress Provider, it is undeniable that Ingress-nginx still occupies the largest share of K8s Ingress Provider, then whether the cloud native gateway can be compatible with Ingress-nginx configuration for users Provide low-cost or even zero-cost migration solutions? With this question, we also did an exploration. Ingress-nginx did not use the method of defining a new CRD to extend the standard K8s Ingress capabilities, but used the method of Annotation. First, we analyzed the Annotation list of Ingress-nginx, as follows:

Prioritized according to user usage alignment. Among them, the Annotation cloud native gateways at high and medium levels already support automatic conversion. In addition to inserting code snippets and other Annotations related to the internal implementation of Nginx, we are also gradually supporting conversion. The goal is to achieve zero-cost migration. A simplified diagram of the conversion process is as follows:

Write at the end

The cloud native gateway provides two types of payment models: post-payment and monthly subscription. It supports the four regions of Hangzhou, Shanghai, Beijing, and Shenzhen, and will gradually open other regions. The cloud native gateway purchase link is here.

You can also nail the search group number 34754806 to join the user group to communicate and answer questions.