If you let me design a set, TPS million-level API gateway!

Author: Xiao Fu Ge Blog: https://bugstack.cn

Precipitate, share, grow, and let yourself and others gain something! 😄

Yes, Brother Fu is going to do something again! This time I am going to start the API gateway project, because this is a core service that all major Internet companies have, and it is followed by Didi Taxi, Meituan Takeaway, Jingdong Shopping, WeChat Pay from users, and even more than 10 million visits during the promotion period. volume of the core system.

🤔 So what kind of project is it? Why does it exist? How is it designed and implemented? What technology stacks are used?

1. Preface: What is a gateway?

In a computer network, a gateway is a server that forwards communication data from other servers. When receiving a request from a client, it processes the request like a source server that owns its own resources.

The API gateway is also a unified communication management system introduced after the traditional huge single application (All in one) is split into numerous microservices (Microservice). It is used to run a traffic entry between external http requests and internal rpc services to implement 协议转换 , 参数校验 , 鉴权 , 切量 for external requests 切量 、 熔断 、 限流 、 监控 、 风控通用服务。

2. Dachang: Why do they all make gateways?

When major factories make gateways, what they actually do is a unified solution. It condenses the common requirements of the same kind from RPC to HTTP communication under distributed microservices into general component services, and reduces the development cost of similar technical demands for non-business requirements under the development of business demand scenarios.

So what to do when there is no gateway in the past, the basic approach is to develop a corresponding WEB service on top of the RPC service, these WEB services can be Spring MVC projects, call the RPC service in the Spring MVC project, and finally provide the HTTP interface to the It can be used in H5, Web, applet, APP and other applications. As shown in Figure 1-1

Several problems in traditional development of WEB services:

• Question 1: Every WEB application needs to match with it and apply for a set of resources such as projects, domain names, machines, etc., until it is deployed, the R&D efficiency is reduced, and the maintenance cost is increased.
• Question 2: Every WEB application will involve some common requirements, such as current limiting, fusing, downgrading, and quantity cutting, and the cost of maintaining code will increase.
• Question 3: Every WEB application will involve the requirements of document maintenance, engineering debugging, and joint debugging during the entire life cycle of use. Development similar to slash-and-burn will inevitably reduce R&D efficiency.

Therefore : In summary, these problems encountered in traditional development under 阿里 have made each big manufacturer have their own demands for self-developed 腾讯 , including; 百度 、 美团 、 京东 、 网易 、 亚马逊等，都有自己成熟的API 网关解决方案. After all, this can reduce communication costs, improve R&D efficiency, and improve resource utilization.

3. Gateway: System Architecture Design

If you want to implement a gateway that can support tens of billions of throughput, then it should be decentralized in accordance with the distributed architecture thinking and support horizontal expansion. Let each gateway service become a computing power, and dynamically allocate different microservice RPC interfaces to each computing power group according to the weight policy calculation, so as to achieve the ability of distributed computing.

In addition, in terms of design and implementation, the communication module, management service, SDK, registration center, operation platform, etc. of the gateway should be developed and implemented separately in turn, so as to be used in independent combination packaging.

This is like why the ORM framework is not strongly bound with Spring during development, but develops an independent component. When the integration of Spring is required, a Mybatis-Spring is developed separately to integrate services.

So it is the same idea when designing the gateway here, just like the communication on the official website should not bind all Netty-related services to the Spring container from the beginning, which increases the maintenance cost and reduces the scalability of the system.

Such software architecture design will be reflected in this gateway microservice architecture. The overall architecture is shown in Figure 1-2.

The core content of the entire API gateway design is divided into five parts;

• 第一块 : It is about the protocol processing of communication, and it is also the most essential processing content of the gateway. Here, it is necessary to use the NIO framework Netty to process HTTP requests, and perform protocol conversion generalization calls to RPC services to return data information.
• 第二块 : It is about the registration center. Here, the gateway communication system needs to be regarded as a computing power. Every time a gateway service is deployed, a computing power needs to be registered with the registration center. The registration center also needs to receive the registration of the RPC interface. This part can be based on SDK automatic scan registration or manual intervention management. When the RPC registration is completed, it will be allocated to a set of gateway computing power by the registration center through AHP weight calculation for use.
• 第三块 : It is about routing service, each registered Netty communication service will be associated with the packet gateway provided by him, for example: wg/(a/b/c)/user/... a/b/c needs to match the Nginx routing configuration to ensure that different interface call requests go to the corresponding Netty services. PS: If the corresponding error or startup, a similar accident at station B may occur.
• 第四块 : Invocation of plug-in modules under the chain of responsibility, authentication, credit authorization, circuit breaker, downgrade, current limit, cut volume, etc. Although these services are not considered as the content under the definition of gateway, they are common services. , they are usually put into the gateway layer for unified design, implementation and use.
• 第五块 : Management background, as a gateway project, a corresponding management background is indispensable, user interface registration and maintenance, mock test, log query, traffic shaping, gateway management and other services.

In summary, the system microservice module structure is as follows:

serial number	system	describe
1	api-gateway-core	Gateway core system: used for network communication conversion processing, undertaking http requests, calling RPC services, and calling responsibility chain modules
2	api-gateway-admin	Gateway management system: used for gateway interface background management, registration and offline deactivation control
3	api-gateway-sdk	Gateway registration component: used for annotation collection interface, sending message registration interface
4	api-gateway-center	Gateway registry: Provide gateway registry services and register gateway interface information
5	api-gateway-test-provider	Gateway test project: provide RPC interface
6	api-gateway-test-consumer	Gateway Test Project: Consuming RPC Interface

4. Demonstration: gateway operation effect

Taking advantage of the weekend holiday, Brother Fu has already done a part of the function realization, just like Brother Xiao used to ["Handwritten Spring"](), ["Handwritten Mybatis"](), this project is also gradually completing each module step by step. function development. And refer to the excellent source-level project architecture design, using abstraction and divide-and-conquer design techniques to solve the coupling call and service design between functions. At the same time, it also combines design principles and design patterns in corresponding scenarios to develop high-quality code that is easy to iterate and maintain. Part of the code implementation and operation is shown in Figure 1-3

• On the left is the API Gateway core communication module, and on the right is the RPC (Dubbo) service. Through the http request initiated on the web page, the result data is packaged and returned to the page through the protocol conversion of the API gateway and the generalization call to the RPC, which is the running effect of the middle picture.
• The realization of the project on the left is completed step by step with progressive splitting modules, such as: core-01 (Netty communication), core-02 (generalized call), core-03 (executor), etc. Interested readers can learn from it; the layering of the architecture, the design of functions, and the implementation of code.

5. Invitation: Let's develop together

💐The above projects related to API gateway are also the content that Xiao Fu is going to take readers to use 周末 and 假期 to learn and practice. Now get on the bus, you will learn with you through the three aspects of Brother Fu's 视频 + 文档 + 代码 to help you improve your technical strength. Renewing your career also means you can go further and make more money.

• [x] Chapter 1: HTTP Request Session Protocol Handling
• [ ] Chapter 2: Proxying RPC Generalization Calls
• [ ] Chapter 3: XML Configuration File Parsing
• [ ] Chapter 4: Method Executor Encapsulation
• [ ] Combing... Weekly update

---

• Github: https://github.com/fuzhengwei/api-gateway
• Gitcode: https://gitcode.net/fuzhengwei/api-gateway