NextRPC : Innovation and exploration of RPC multi-segment return

Author: Wu Xinwei (Story Ang)

NextRPC is an innovation and exploration of the RPC request mode. Like a multi-stage rocket, it can return multiple segments of payload data, request/reply multiple segments from different network channels, and finally complete the delivery of business scenarios. The "Next" in it has a double meaning:

1. The core difference is that the request is different from the traditional RPC. The feature of the request is that the request response is returned to the client in multiple segments. In the streaming semantics, the data is continuously responded through the Next -> Next -> Next mode;
2. Next means the next generation, and NextRPC is the next-generation RPC mode of traditional single-response.

At present, NextRPC has been launched in some transaction scenarios of Taobao. For example, the order redemption business will be launched in early April 2021. After two major promotions on 9.9 and 11.11, the business model has stabilized. In the Double Eleven promotion in 2021, the NextRPC link will bring more than 5% increase in uv conversion to the business as a whole. In the scenario where one optimal product is revealed among multiple products, the uv conversion will increase by more than 25% .

business background

In some transaction core links such as shopping carts, orders and other scenarios, it is expected to introduce a recommendation algorithm for shopping guide links, based on the store dimension to make personalized recommendations for single product, multi-product SKU, and cross-store scenarios. The disclosure of personalized recommended products or other preferential information to improve the UV conversion rate.

Issues and Challenges

When the core transaction scenario encounters personalized recommendation, transaction and shopping guide scenarios collide, the following new problems will arise:

• Conflict of user experience: personalized recommendation algorithm service RT is high, and does not meet the RT requirements of core transaction links;
• Conflict of service quality: The introduction of personalized recommendation will make the downstream system of the transaction link more complex, which will bring new challenges to the stability of the system; at the same time, the "personalized recommendation business" of the shopping guide link can tolerate partial uncertainty (such as request timeout/failure), while the "core transaction link" requires extremely high stability and certainty, and each failure may result in a missed transaction;
• Conflict of machine resources: The multi-location deployment structures of transactions and shopping guides are different, and there are differences in machine capacity and distribution.

The above problems lead to new technical challenges: to support multiple business support systems with different requirements for "user experience", "service quality" and "resources" in one request.

Technical selection

1. RPC mode analysis

The following is a comparative analysis of five common RPC models:

2. Analysis of request processing model

The request processing model mainly analyzes two models of serial processing and parallel processing.

• Serial processing: The dependency depth of data determines the overall minimum rt that can be optimized;
• Parallel processing: reduce the rt on the corresponding level to the original max(RTn) through parallelization (concurrency n).

3. Multi-segment return mode of NextRPC

NextRPC combines the RPC mode of single-request asynchronous push data stream and the request processing model of parallel processing to solve the challenges brought by new business scenarios:

• Asynchronous push data stream with single request:

- It can decouple core business logic and non-core business logic, ensure synchronous response of core data and asynchronous response of non-core data, and solve service quality problems;

- Logical decoupling of transaction links and shopping guide links, internal cross-application calls to solve machine resource problems;

• Parallel processing:

-Business logic can be processed in parallel, saving serial waiting time and solving the problem of user experience in transaction links.

NextRPC Architecture

1. Client Architecture

Technology Architecture

• Network layer, smooth Mtop/Accs channel, transparent to business;
• The data rectification layer controls the timing of multiple returns of a single request (the asynchronous sub-return contains asynchronous business data flow);
• The data orchestration layer controls the merging of asynchronous data between multiple requests.

Dataflow and multi-instance

• One page supports multiple NextRPC instances, and multiple pages can create multiple NextRPC instances;
• A NextRPC instance can make multiple different requests and receive different side response pushes.

2. Server architecture

Technology Architecture

Detailed request process

1. The client initiates the main request;
2. The server receives the client's main request, and at the same time executes the business logic, it sends the secondary request information through the message middleware (Note: AttachedRequestEmitter decouples non-core dependencies through the message middleware), after all business logic is executed, the secondary request are also sent;
3. Return the client main response, the main response contains the secondary request meta information;

4. The secondary request receiver consumes the secondary request message through the message middleware, and calls the service processor AttachedRequestProcessor. After the service processing is completed, it returns the nextrpc-sdk secondary request processing result;

   1. The secondary response is sent to the client through the message push channel
   2. The client listens to the specified message push channel, and after receiving the secondary response data, renders it on the screen.

exception handling

1. Sub-request orchestration, in a scenario where a main request has multiple sub-responses, how does the client know how many sub-requests there are in total?

Since the server business calls the NextRPC SDK to send a sub-request, the SDK knows how many times it has been called, so this information is counted and stored by the SDK. The SDK internally maintains a sub-request meta information, including the total number, sub-request business information, business Each time the SDK is called to send a secondary request, the SDK accumulates the total number. When the main response ends, the total number of secondary requests is sent at the same time.

2. In the processing of secondary requests, the throughput of different services is different. How to isolate multiple services to ensure that they do not affect each other?

Businesses are isolated by different topics or different tags of the message middleware, and each topic or tag is consumed by a separate thread pool.

3. How to deal with expired and repeated messages?

The secondary request is sent through the message middleware. When the receiving end of the secondary request receives the message, the message is expired or duplicated. The SDK provides the meta information of the message (including timestamp and unique identifier).

Message expired: SDK internal

Message duplication: idempotency needs to be implemented by the business code based on the unique identifier of the request.

3. Technical indicators

NextRPC, as a single-request multiple-response RPC mode, is divided according to the request stage, and the technical indicators for measuring service quality can be divided into three types:

• Main Request Success Rate: The percentage of main requests that are successfully processed. The success of the main request determines whether the subsequent process can continue, and it is also the most basic indicator to measure the service quality of NextRPC;
• Sub-request success rate: The ratio of sub-requests from being initiated to being processed successfully, which is used to measure the service quality of sub-requests;
• Secondary response arrival rate: On the basis of the success of the main request, from the initiation, reception, processing of the secondary request, to the successful reception of the secondary response by the end device, it is the core measure of the service quality of NextRPC's multi-response. With the timeout configuration of the request, the arrival rate of 1s, 3s, and 5s can be further measured.

Different request stages will have different technical indicators, which can be used for client and server service quality analysis. The indicators produced in each stage are shown in the following figure:

Summary and Outlook

Double 11 is a super project of Alibaba, and 2021 is the thirteenth year of Double 11. Based on the technological and commercial innovation of the past 12 years, it is the beginning of a new cycle.

From the pursuit of higher to the pursuit of better, technology is also exploring this process. Focusing on the goal of not degrading the consumer experience, we have made bold technical attempts in the core transaction link. While pursuing transaction certainty and experience, we provide better Better and higher-quality services satisfy consumers and empower businesses to improve their operational capabilities.

In the future, we will continue to try to implement more business scenarios, and use NextRPC to help business innovation!

are hiring!

Welcome to Taobao Mobile Technology Center. There are many team members, including founder members of Alibaba Mobile Middleware, core members of the Eagle Eye full-link tracking platform, and a group of small partners who love technology and hope to use technology to promote business.

Taobao Mobile Technology Center, promotes the upgrading of the Tao Department (Taobao, Tmall, etc.) structure, and is committed to providing basic core capabilities, products and solutions for the Tao Department and the entire group:

1. High-availability solutions and core capabilities for business (application high-availability: providing flexible high-availability solutions for adaptive current limiting, isolation, and fusing for businesses, site high-availability: fault self-healing, multi-room and remote disaster recovery, and rapid flow switching recover);
2. A new generation of business research and development model FaaS (one-stop function research and development Gaia platform);
3. Implementation and implementation of the next-generation network protocol QUIC;
4. Mobile middleware (API gateway MTop, domain name scheduling AMDC, message/push, file upload AUS, mobile configuration push Orange, etc.).

Looking forward to participating in the construction of the basic platform of Taoxie~

Resume submission to : [e-mail]

Follow [Alibaba Mobile Technology] WeChat public account, 3 mobile technology practices & dry goods per week for you to think about!