Billions of visits per day! Interpretation of the high-energy evolution of a push API gateway

Recently, Li Bai, a push server technology expert, was invited to participate in the SegmentFault D-Day online technology live broadcast event to explore the "back-end architecture evolution" with back-end technical experts from leading Internet companies. With the theme of "The Evolution of API Gateway", Li Bai shared his practical experience and in-depth thinking on the construction of API gateway based on golang.

★The following is Li Bai’s speech and dry goods:

Origin of API Gateway

API gateway is an architectural pattern that emerged with the concept of "microservices". In the process of microservice splitting, the originally huge monolithic applications and business systems were split into many microservice systems for independent maintenance and deployment, which led to the multiplication of the scale of APIs and the increasing difficulty of API governance; at the same time, the subsystems passed APIs When providing external capabilities, there will also be the problem of repeated construction of general capabilities. Therefore, the use of API gateways to publish and manage APIs in a unified manner has gradually become an architectural trend.

In this push, the company's core business systems such as news push and financial risk control are all based on a microservice architecture, and some systems also have self-built gateway modules. With more and more dependencies between different systems, the need for efficient interface governance is becoming increasingly urgent. Therefore, I Push introduced a unified API gateway very early to solve a series of problems such as permission control, flow control, service degradation, gray-scale release, and version management.

1. Push the early API gateway

In 2015, the birth of SpringCloud greatly promoted the development and popularity of microservice architecture. It was also during this period that I relied on SpringCloud gateway to build its own real API gateway.

SpringCloud is ecologically friendly. There are many monitoring and fault-tolerant components that can be used out of the box, but because it does not support traffic and security governance, it cannot well meet the needs of subsequent companies for API governance; especially in the construction of data after a push Taiwan, a unified API gateway is urgently needed to bear the ingress traffic of the data center. In addition, in terms of development language, SpringCloud only supports Java, which has limited applicability; and has disadvantages such as unsatisfactory performance and difficult operation and maintenance.

Therefore, in order to achieve stronger API governance capabilities, simplify access and operation and maintenance methods, and also to better adapt to the company's data center construction needs, I recommend choosing a self-developed API gateway.

2. A self-developed API gateway

2.1 Self-study goals

Several key goals for building an API gateway:

✦ 1) To be able to manage the complete life cycle of API

For example, in the API design, it is necessary to unify the specifications, stipulate that the API must have attribution services and tags to achieve isolation in the API design; after the design is completed, it must be directly debugged and automatically generated test code; when it is released, the API traffic must be done Refined control to support the gray-scale release of services; in the process of API operation, it is necessary to be able to monitor and alert the API call in all directions, to timely fuse and isolate the services that have problems; to recover resources in time after offline.

✦ 2) Complete functional components are required to handle the request process

In the entire request link, we designed and implemented a series of core functions such as link tracking, logging, authentication, current limiting, fusing, and plug-ins.

✦ 3) While ensuring the "three highs", user migration and access should be simple

This actually means that the API gateway should be easy to operate, maintain and use, and at the same time provide various indicator detection functions, and support automatic fault tolerance and elastic expansion.

2.2 Technical selection

Based on the above target design and technical research, we chose to use golang as the main development language for the self-developed API gateway. There are several reasons for choosing golang:

✦ 1) In the process of multi-computer room disaster recovery construction and data split migration, gproxy-codis and gproxy-es have been designed, and a set of proxy clusters have been built to route different users. These proxies have been running well so far, and some large clusters have single-machine QPS exceeding 6W. In the development process of these projects, we have accumulated a lot of development experience and basic components, many of the wheels can be directly reused. Therefore, we use golang and based on these existing components, developing a set of gproxy-http is relatively labor-saving.

✦ 2) Golang language itself naturally supports high concurrency, fast development speed, and the most important thing is to save machine costs.

✦ 3) Golang is the most used language as a cloud native framework, and the technical precipitation on golang is also paving the way for the promotion of cloud native construction.

2.3 Design and implementation

After confirming the target and technology selection, the next step is to design and implement some concrete.

✦ 1) Overall architecture

The overall architecture design of a push API gateway is shown in the figure:

The first is a web management platform. The creation, release and subsequent management of APIs can be configured on the management platform; after the configuration is completed, it will be sent to the configuration center to notify the gateway, and the gateway will also periodically pull the full amount of API configuration; Then there are some core components of the gateway, such as the plug-in engine, which are mainly some plug-ins that perform configuration. The forwarding engine is also the core module of the API gateway. A push forwarding engine supports http, grpc, and a self-developed gcf protocol. In the data center business scenario, it also supports the data push capability of Kafka.

✦ 2) Plug-in service

As you can see from the figure, there is an independent plug-in service in the overall architecture of the push API gateway. The core reason for this design is that golang is a C-like language, packaged as an executable file, and golang's native plug-in is directly compiled and does not support updating and uninstalling, so it is not possible to directly add or update plug-ins on the interface. For this reason, we have developed a plug-in service using Java, using Java's dynamic language features to flexibly support the addition, update, and uninstallation of plug-ins.

The gateway communicates with the plug-in service through grpc, and there is a certain loss in performance. In order to reduce the performance loss as much as possible, we use a native plug-in of golang to implement encryption and specific serialization related plug-ins. For some business-customized components, it is recommended to use Java plug-in services.

✦ 3) Resource isolation

Resource isolation is a common method to achieve high availability of the system. The isolation design mainly includes the isolation of clusters and thread pools.

The API gateway mainly supports service cluster isolation. Through this cluster-level isolation, multi-tenancy can be supported at the upper layer. If it is more thorough, the gateway cluster can also be isolated at the LB level. In addition, the gray-scale release of services is also realized through the cluster isolation of the gateway. The specific process is that during the upgrade, the user can configure traffic forwarding rules and clusters on the interface, and import part of the test traffic to the gray cluster through traffic playback, or forward the real online traffic to the gray cluster in proportion to ensure that there is no Full release after the question.

The isolation of threads is mainly reflected in the data service. The main function is to make the data API, that is, the data of MySQL, ES, Hbase can be provided through the API through a simple configuration in the interface, without the developer writing CRUD or client code ,Very convenient. At present, most of the traffic in the platform business scenario of a push data is requesting data services, so we have designed three types of thread pools: ordinary thread pool, slow thread pool, and custom-configured thread pool. When the request duration exceeds the slow threshold, the interface will be allocated to the slow thread pool for processing to prevent slow requests from dragging down the entire service.

✦ 4) Service orchestration

Service orchestration is also a common requirement that API gateways need to meet, mainly to aggregate multiple API calls to greatly reduce call latency. This part of the function was previously on the gateway. The service orchestration supported at the gateway level is relatively basic. It can support concurrent aggregation calls of APIs, but it is difficult to handle complex business combinations, especially when it comes to the orchestration of transactions. Therefore, we decided to extract this part of the function as an independent service, and the subsequent link gateway will directly access the service orchestration module, which also ensures that the overall gateway is relatively lightweight.

✦ 5) Performance optimization

For API gateway performance, we have also made a series of stress tests and optimizations. For example, we use open source functions to replace or rewrite the serialization, encryption and decryption functions that are used extensively in the internal; use Sync.Pool to reuse objects to make their internal logic Perform pure asynchronous processing; self-developed gnet, replace the native net framework, optimize the network model, etc. Judging from the actual online operation results, in the current push data, Taichung's API platform is called more than 1 billion times a day, the peak QPS of a single machine is about 2W, the overall performance loss is 10%+, and the performance exceeds expectations.

✦ 6) Easy-to-use design

Through the above plug-in mechanism, isolation means and extreme performance optimization, we ensure that the API gateway platform built is highly available and easy to expand. After the platform is completed, it must be convenient for users to access, use, and maintain.

Therefore, in order to improve ease of use, we adopt a pure Web interface design and have built-in multiple API templates. The user can create an API through simple configuration, such as the authorization period of the interface, QPS, quota and other authority configurations, which can be completed through the visual interface operation; after the API is created, the user can also debug directly on the interface.

At the same time, in the scenario of externally providing APIs, users can export API documents under a certain service in batches, which is very convenient. The push API platform also implements monitoring and statistics functions, such as providing data such as the trend of API calls, the number of API calls under the entire service, and error statistics, which is more friendly to operations and R&D personnel.

Summarize

In summary, the individual push API gateway is based on golang's independent research and development, fully web-based configuration, and realizes the standardization and visualization of all API interfaces; in addition to solving the basic requirements of the gateway, it also supports plug-in hot update, multi-protocol conversion, data push, and cluster level. Advanced requirements such as resource isolation. In addition to being integrated into the system's microservice architecture, the personal push API gateway also serves as an entrance to the company's data center and Taiwan, with an average of billions of visits per day. At present, the new version of the API gateway cluster has been running stably for more than a year, and it is still evolving while ensuring the stable operation of the system. In the future, we will also explore more possibilities around cloud native and provide a stable and efficient basic platform for our business.