A brief analysis of Alibaba's evolution in Envoy Gateway

Author: Geng Leilei (Ru Feng): Alibaba Cloud R&D engineer. From May 2020, he was responsible for the construction of Envoy Gateway to the launch of 3.0. As the technical leader, he led the entire evolution process and has rich practices in the field of cloud native gateways.

I recently read the article "The Envoy Gateway is here", and I deeply feel the powerful scalability of Envoy and the ease of use based on the Envoy Gateway. Under the K8s architecture, Envoy redefines the positioning and capabilities of the gateway, and is known as the cloud. Native gateways are even called next-generation gateways. Alibaba started the exploration of next-generation gateways as early as 2018. This article will give a brief introduction to this exploration process.

As early as 2018, Alibaba started the prelude of cloud native cloud migration, using containers and service grids as core technology points for evolution, and tried Alibaba and Ant to unify their middleware technology stacks through this technological evolution. , so that the business can focus more on business development and shield the underlying distributed complexity. As an important direction of service mesh, we have started the exploration of next-generation gateways.

Envoy Gateway 1.0 (incubation period)

In the process of going to the cloud, we expect to unify the application architecture technology stack, but the RPC protocols of Ant and Alibaba are different. There are long intermodulation links, high protocol conversion consumption, and loss of Tengine Reload access (the access needs to be continuously reloaded as soon as the access takes effect. If the impact of reload is controlled, the number of reloads should be reduced, the access service will take effect slowly), and the Nginx kernel service governance capability is weak. Therefore, a future-proof gateway solution is required.

At that time, we had two technical evolution ideas, one was based on Tengine for optimization, and the other was based on Envoy kernel to expand gateway scenarios. Considering that Tengine’s solution to these scenarios changed too much, Envoy, as the second option for gateways, could simply solve the problem. The above pain points, therefore, we chose the Envoy kernel as the next-generation gateway evolution direction, and from the statistics of CNCF Ingress Provider, Envoy is also the fastest growing and highly accepted by the community.

In May 2020, we launched the research and development of Envoy Gateway 1.0, which successfully supported the Double 11 promotion in the same year, and became the core business link for reinsurance.

Envoy Gateway 1.0 is mainly used for RPC interworking of east-west traffic. Its architecture is deployed as follows:

During this period, we have evolved the triple protocol of Dubbo3.0 in the future, and based on Envoy, we have evolved the service management capabilities of the gateway, supporting the traffic peak of hundreds of thousands of TPS in the local life campaign of Double Eleven that year.

Envoy Gateway 2.0 (growth stage)

With the advancement of Alibaba's cloud campaign, more and more scenarios have found us, such as the interoperability of services on and off the cloud. Due to the weak service management of Tengine, a large number of second-tier microservice gateways within Alibaba need to converge. Therefore, from a business perspective, we need to Do the evolution of the Tengine+Envoy two-layer gateway and undertake the north-south gateway traffic. In December 2020, we started the evolution of the 2.0 architecture. The following is an example of the Youku scene to illustrate the evolution process as shown below:

The north-south architecture diagram of Envoy Gateway 2.0 is as follows:

In the two-tier architecture, the Envoy gateway is more responsible for the needs of microservice gateway and microservice governance, and is integrated with the Tengine traffic gateway. In this process, we have improved service governance and high availability, and supported the unification of multiple second-tier microservice gateways within Youku, greatly improving performance and O&M efficiency.

In the 2.0 stage, Envoy Gateway completed the scheduling and distribution of east-west and north-south global traffic. East-west not only supports ant RPC intercommunication across business domains, but also extends to hybrid cloud on-cloud and off-cloud RPC intercommunication scenarios, including DingTalk documents , Alibaba Video Cloud, Dharma Academy's store Xiaomi, Smart Digital Human, etc., the business picture of the 2.0 stage is as follows (the scenario of intercommunication between the cloud and the cloud, using DingTalk as an example):

With the rapid development of the Envoy Gateway business, when we continued to cooperate with Youku, everyone raised a question: Can the two-tier gateways of Tengine Gateway (acting as a traffic gateway) + Envoy Gateway (acting as a microservice gateway) be merged? Use Envoy Gateway? The answer is yes, and we have also collaborated to design a new architecture diagram, as follows:

The evolution of this solution allows us to see the new development trend of gateways, especially in the context of K8s-led containerization, the natural isolation of the network inside and outside the K8s cluster, users also need a product that takes into account high performance and security, as well as powerful services The entry gateway of governance capabilities, which also provides a good accumulation for us to move towards 3.0.

Envoy Gateway 3.0 (mature stage)

With the polishing of a large number of scenarios in Alibaba, the performance and stability of the Envoy gateway have achieved good development. In 2021, Alibaba launched the middleware trinity battle to support the group's business with cloud products. Therefore, we will also incubate mature technologies to serve the group through the MSE cloud native gateway.

Click here for more details on MSE

At this time, while we combined the traffic gateway + microservice gateway into one through Envoy, we also continued to optimize the resource deployment cost of the gateway through hardware acceleration, kernel optimization and other means without sacrificing performance.

The technical architecture determines the technical advantages. Envoy's natural scalability can also integrate rich security authentication and microservice governance capabilities, reflecting the advantages of high aggregation of cloud native gateways, such as:

• The gateway is directly connected to the service PodIP, without the traditional Cluster IP, and the RT is lower
• Support HTTPS hardware acceleration, QPS increased by 80%
• Support Wasm plug-in market, plug-in hot-loading, meet the needs of multi-language custom plug-ins
• Self-developed Multi-Ingress Controller component supports multiple cluster Ingress multiplexing the same gateway instance
• Natively compatible with K8s Ingress specification, and supports seamless conversion of Nginx Ingress core function annotations

Give back to the community

During the evolution of Envoy Gateway, we also raised many community issues, including: dubbo_proxy, wasm, cryptomb, etc. In the future, we will continue to give back to the community, make more contributions, and work with the community to build the next-generation gateway.