SOFAServerless system facilitates rapid business development

Text｜Zhao Zhenling (flower name: Youji) Ant Group technical expert Liu Jing (flower name: Fei Lian) Ant Group technical expert The following content is compiled from the sharing of the fourth anniversary of SOFAStack

The SOFAServerelss R&D operation and maintenance platform is developed by Ant Group to help applications iterate quickly and steadily under the circumstances of business development and the increasing complexity and cost of R&D operation and maintenance. A set of solutions evolved from the perspective of refining R&D operation and maintenance granularity and shielding infrastructure.

The core method is to split the application from the code structure and developer formation into two levels through class isolation and hot deployment technology: the business module and the base. The base provides a computing environment for the business module and shields the infrastructure, and module developers do not. Perception of infrastructure such as machines and capacity, focusing on business research and development to help business develop rapidly.

background

The current development of serverless has two evolution directions, one is the evolution from the function computing-oriented architecture to the online application, and the other is the evolution of the online application-oriented architecture to the function-like computing direction.

The SOFAServerless system chooses the latter, which is based on some problems encountered in the process of application R&D, operation and maintenance. In the field of application architecture, the inevitable problem is that as the complexity of the application continues to increase, problems in the process of R&D, operation and maintenance will continue to be exposed.

First of all, let's take a look at what is the process of R&D and operation and maintenance for ordinary applications?

As shown in the figure, from requirements to design, development, offline testing, and then to the R&D operation and maintenance on the release line, it is a process of continuous feedback and loop iteration. It can be simplified as developers submit code to the code repository, do parallel verification tests offline, and publish online after passing the test. The publishing process is serial, and there can only be one publishing window. Such a process is not yet available in the application volume business. The problem is not very obvious when it is too complicated.

However, when the complexity of the business continues to increase, some new problems will appear in the iterative process of ordinary applications, as shown in the following figure:

1. High management costs: requirements management, code management, personnel management
2. High time cost: online verification and publishing block each other; single startup is slow
3. High risk of change: one change involves all code; one change involves all machines
4. Not scalable enough

In addition, since these problems are caused by the coupling of multiple business units and R&D tasks at certain single points, the costs of these R&D operations and maintenance show an exponential increase with the complexity of the business.

SOFAServerless R&D operation and maintenance system solution introduction

For these problems, the industry has developed and evolved a variety of application architectures, from monolithic architecture -> vertical architecture -> SOA architecture -> distributed microservice architecture -> service mesh architecture, etc. The problem of R&D operation and maintenance brought forward the SOFAServerless R&D operation and maintenance system. The main core ideas are:

1. Refinement of R&D operation and maintenance efforts <br>Through refinement, multi-person collaborations do not block each other; the scope of iteration becomes smaller and the speed becomes faster.

2. Shield infrastructure <br>Shield infrastructure, so that business development students only focus on code services and traffic.

For these two points, we use SOFAArk ClassLoader class isolation and hot deployment capabilities to split the application into bases and modules.

Pedestals and Modules

From this picture, we can see the form of our split, splitting a common JVM application into multiple modules, and further dividing the modules into some divisions of labor: base and module, and the corresponding R&D personnel are also divided into base developers and module developers.

The base precipitates general logic, provides computing and environment for modules, and shields the infrastructure for module developers, so that module developers do not need to care about capacity, resources, and environment. Each module is an independent code repository, which can conduct independent R&D operation and maintenance, so that the granularity of R&D operation and maintenance is refined, and because the base shields the environment and infrastructure for the module, module developers can focus on business development and improve overall efficiency.

How to share and communicate

If the application is only split, it is an incomplete solution without sharing and communication capabilities, and it is difficult to solve practical problems. For sharing and communication, the base as a shared layer can help modules to warm up the RPC database cache common classes, common methods, and common logic, which can be installed for some modules to be reused. In this way, the implementation of the module is relatively light, so the deployment density of the module can also be made very high.

For module communication, the current communication between modules can support any communication mode, such as base-to-module, module-to-base module, and calls between modules. Since the module communication is a cross-ClassLoader call within the JVM, the overhead of serialization and deserialization is increased with the method call within the common JVM. At present, this part of the overhead has been optimized to be approximately equal to the method call within the JVM.

After this capacity building, the cost of module access transformation can be greatly reduced and the applicable business scope can be expanded.

How to solve business pain points

management costs

Compared with the original R&D model, the R&D personnel are divided into different groups, and the code warehouse is also divided into multiple module warehouses, which can be independently published online in parallel, and the entire pipeline can be carried out independently.

In this way, the cost of requirement management, code management, and personnel management will be reduced, and there will be no mutual blocking problems in the online publishing process.

Of course, these cost reductions do not mean that these problems are completely gone, but it has changed from the original exponential growth to this linear growth. As the complexity of the business continues to increase, its benefits will become more obvious.

Time costs

Compared with ordinary applications, it takes 3 minutes to build an image, and it takes about 3 minutes to download, start, and mount the image for publishing, and it takes an average of 6 minutes in total; it only takes 10 seconds to build a module, and about 1 to 10 seconds to start (the size of the module can be large or small). Small, for smaller modules, the speed can be done in milliseconds).

The time required for a release has been reduced from the original 6 minutes to 15 seconds, and an iteration has been reduced from the original 2 weeks to 2 days, and the fastest can be launched in 5 minutes.

Scalability

For the deployment form of online clusters, the modules deployed on different machines are not the same. For example, for module 1, which is only installed on the first and second machines, only these two machines are involved in the module upgrade, and the scope of the changed machines is relatively small. In addition, if module 1 needs to be expanded, you can select idle machines from the cluster for hot deployment of the module, which is generally at the 10s level, so it can increase the rapid horizontal expansion capability.

change risk

For a module upgrade and change, only the code of the module itself is involved, not the entire application code. The machine that needs to be updated when the module is changed is only the machine where the module is installed, and does not involve the entire cluster. Therefore, the scope of change is greatly reduced, and the risk of change can be significantly reduced compared to ordinary applications.

High availability and matching capabilities

On the basis of solving the pain points of business R&D, operation and maintenance, the SOFAServerless system has built high-availability and supporting capabilities.

Resource isolation

Resource isolation is reflected in a single JVM. Here we use AliJDK's multi-tenant isolation capability within our company. Each module can specify its own upper limit of resource usage.

For example, there are some problems in the logic of one of the modules, which consumes a lot of resources and will not affect other modules, which is equivalent to the isolation of faults.

Traffic isolation capability

For a single cluster, we have done some fine-grained traffic routing. The main reason is that tags can be dynamically added when sending services, and some rules can be configured when traffic routing is pushed to MOSN and Layotto, so that traffic can be routed according to the corresponding tags. isolation capability.

Observability capabilities and change defense capabilities

It has module-level health check, resource monitoring, log monitoring and troubleshooting capabilities, and builds module-level change defense on this basis.

A module can have multiple versions at the same time, and can do some quick A/B tests, grayscale, and rollback capabilities.

business form

SOFAServerless has developed to the present, and more than 700 Java and nodejs applications have been connected to Ant, which basically covers all business lines of Ant, and has supported more than 10,000 complete production and R&D iterations. Offline can be released in seconds. Many businesses in the Alipay app run on SOFAServerless, such as booth placement, public welfare games, and marketing gameplay.

Last year, SOFAServerless successfully supported heavyweight promotions and events such as 618, Double 12, and Wufu, and withstood the test of high-traffic and high-concurrency scenarios. The peak value of resources hosted on SOFAServerless was around 220,000 cores.

SOFAArk has two core competencies:

1. Divide the application into finer-grained bases and modules <br>Independent life cycles, decoupling R&D, operation and maintenance operations, and improve collaboration efficiency.

2. Separate "code deployment" and "service registration"
A concept similar to FaaS trigger is implemented, that is, you can install modules first, but do not publish any services, but receive instructions at runtime to dynamically complete the publishing/deregistration of services. The whole process does not require code changes, application restarts, and only takes time. a few seconds.

In this way, the "service" itself has become an independent, flexible and lightweight operation and maintenance unit. The business can quickly "split" the service as needed, and conduct more refined governance around the "service". The source is split into multiple small services for isolated deployment, or some secondary and offline services are split from the original application and deployed to a cluster to avoid affecting the online formal business. If the R&D operation and maintenance model based on the previous "application" is quite cumbersome to achieve the above effect, it may also involve code changes, but now the cost is greatly reduced, and business students only need to simply configure it on the interface.

Based on these two capabilities, the landing forms of different businesses are different. According to our experience, there are generally three types, from simple to complex: "Code Snippet", "Module Application" and "Mid-platform Business".

There are big differences in the division of responsibilities and the R&D process.

code segment

In this form, the module is very simple. In a minimalist case, there may only be a small piece of code and a class that carries a small piece of computing logic. The base exposes a unified interface to the outside, and all traffic will be carried by the base after it comes in, and then distributed to different modules for execution. At the same time, the base also provides some general low-level capabilities for modules to call. This form is close to the current mainstream FaaS products, and is suitable for businesses with relatively simple forms, such as computing business and BFF. Its R&D process is relatively simple, and it may not even have the concept of iteration. It can be changed and tested at any time and released immediately. .

module application

As the name implies, each module independently carries a slightly complex and relatively independent business, and provides services directly to the outside world. The base generally does not expose services, and only provides basic capabilities for upper-level modules. Modular applications are suitable for multiple small businesses in the same business domain, and a large number of underlying capabilities can be shared. The R&D model is similar to traditional applications, except that the R&D object becomes a lightweight module instead of a large application. The R&D process between different modules is completely decoupled, avoiding collaboration problems such as release card points, waiting, and environment preemption.

Middle-end business

Modules do not directly provide external services, but only atomic components, which are implementations of extension points exposed by the base. The base will undertake all traffic, expose services through a unified external interface, and then serially arrange the components in the module after receiving the call to complete a complete execution of business logic.

The middle-office business is the most complex form at present. On the one hand, it requires modules to be released in groups for operation and maintenance. The business and the module are in a many-to-many relationship. The research and development process involves the simultaneous release of multiple modules, and the simultaneous release of one module to multiple businesses. , it is necessary to design the relevant process experience; on the other hand, it uses the characteristics of dynamic splitting services, and different businesses publish services independently based on the same set of interfaces provided by the base.

Case-Module Application-Social Game

Social games are a typical case of module applications. Modules carry different mini-games and have different life cycles. In business, they can quickly and frequently iterate through trial and error without affecting each other. The common logic and infrastructure of different games sink to the base, such as common models, unified storage dependencies/downstream dependencies, event-driven frameworks, etc., which are relatively stable and slow to iterate. At the resource level, each mini-game has an independent cluster deployment, and Pods in the cluster will not install other modules.

Module applications are relatively simple in terms of R&D and operation and maintenance.

Case - Middle Office Business - Marketing Play

Marketing gameplay is a typical middle-office system. It is responsible for marketing the daily and big promotion marketing activities in the entire Alipay APP. The overall structure is as follows:

From the bottom to the top, in addition to providing general services related to marketing, the core of the base is the process engine, which locates the corresponding process template according to the business attributes of a call, and arranges the execution of components in the module. The modules are organized according to the gameplay and provide atomic components, which are developed by different business teams. The general modules are maintained by students in China and Taiwan, and provide general components that may be used in different gameplays. In this way, the modules become very lightweight. After construction The Jar is less than 1M, and the startup time is within 5s.

Finally, the upper-layer business combines multiple modules as required, and publishes its own services through the unified interface of the base.

Taking "PlayTrigger" as an example, the components it needs are provided by the power play module and the general module. At the same time, based on the interface provided by the base (PlayTriggerFacade), an RPC service marked with the "Power Play" tag will be released. To be more specific, when we released the business of "Assistive Gameplay", it can be simply understood as installing the two modules of the Assistive Gameplay and the general module on the base, and then pushing the instruction to the base, and publishing the "Assisted Gameplay" label. RPC service.

Once we divide it according to business at the R&D and service levels, we can easily do resource isolation and resource scheduling between businesses.

The online business is divided into two business clusters. The "Internet Business Banking Cluster" only installs the modules related to the online business assistance game, and publishes the services of the online business assistance game.

At the same time, we implemented the business deparameterization and service routing capabilities in the sidecar (MOSN) of the upstream application, so that the unified PlayTriggerFacade interface is seen when the upstream is called, without any code modification. However, in the end, according to the configured business rules in MOSN, the traffic is correctly routed to the corresponding cluster in the game center.

Under the Ark technology stack, resource mobilization between clusters is a very lightweight operation. If you want to move the Pod from the "Internet Merchant Bank Cluster" to the "Daily Marketing Cluster", you don't need to restart the process. You only need to uninstall the modules and services on the Pod into an empty base, change the ownership relationship between the Pod and the cluster, and then Just deploy new modules and services, and the whole process takes less than 10s. If you really need to pull up a new Pod from 0 to 1, we also provide a buffer pool to bypass the time-consuming restart of the base. The resources of the cluster shrinking will enter the buffer pool first, and will not be destroyed directly. Other clusters can be expanded when they are expanded. Borrow resources directly from the buffer pool.

At present, cluster scaling requires manual decision-making and triggering, which is not intelligent enough. This year, we will focus on building capabilities such as automated elastic scaling, non-peer-to-peer deployment mode, and machine cold-start optimization, so that businesses can only focus on the number of instances and scaling strategies of services and provide a more serverless experience.

During the Double 12, China and Taiwan were fully connected to SOFAServerless. From the start of the big promotion to the end of the event, related modules were released 15 times online and 737 times offline, with an average time of less than 10 seconds. , and it can be tested immediately after delivery”, which is a great improvement for the joint debugging of business development. At the same time, only the modules related to the Double 12 gameplay have been changed during the entire R&D cycle, which has no effect on other gameplays.

Summarize

Finally, briefly summarize the core technical advantages of SOFAServerless compared to other serverless products:

1. Low migration cost <br>Ordinary SOFABoot and SpringBoot applications only need to add some starter dependencies to access the SOFAServerless system, with the ability to hot deploy modules and dynamically publish services, and if you want to migrate stock Java applications to other FaaS products , will face a huge transformation cost;

2. The form of multi-module combined deployment in the process is more suitable for expressing complex business logic

3. Second-level R&D deployment, almost zero increase in communication overhead, higher deployment density and lower cost

Compared with other serverless products, we have realized the combined deployment of multiple modules in the process, and did not adopt the method of 1 Pod 1 module. The advantage is that the modules are in-process communication, and there is no communication overhead. After splitting, a call may involve modules, Dozens of calls between bases. If you use cross-process communication or remote RPC between Pods, the overhead is unacceptable. In addition, higher deployment density can be achieved, and some long-tail, low-traffic services can be intensively deployed on the same batch of Pods at lower cost;

4. Low-cost and refined traffic isolation, routing has no sense of upstream <br>The ability to dynamically publish services can be used to break down the original coarse-grained services into finer details without changing the code, and achieve business resource isolation at low cost. Govern traffic more finely, and all this is insensitive to upstream, without any modification.

SOFAArk About Open Source

The SOFAArk 2.0 framework has been released. Compared with 1.0, we have made major upgrades and optimizations in the ClassLoader system, runtime performance, startup speed and many other aspects. Interested students can visit the GitHub repository to read the source code.
https://github.com/sofastack/sofa-ark

At the same time, the components related to the entire R&D operation and maintenance system of SOFAServerless are gradually being open sourced. We will launch the first open source version in October, including the Serverless management and control platform and the Ark Scheduler system.

We are also actively embracing the open source community. The open source version will support the release operation and maintenance capabilities of SpringBoot application modular hot deployment, and can connect to the base application deployed on the native Kubernetes Deployment.

Finally, I hope everyone will actively participate in the open source capability building of the SOFAServerless technology system, and help industry applications smoothly open up the Serverless R&D experience!