SAE Best Practice Template: Helping Vision Digital Enter the "Fast Lane" of Cloud Native

Author: Hu Yang (Vision Mathematics)

In 2021, the business value of cloud native is being released at an accelerated rate.

It is a recognized fact that serverless is the absolute bright spot in the current cloud native direction. It can be seen that its emergence allows enterprise users to truly relieve the burden of operation and maintenance and focus more on the business itself. In other words, enterprises can reduce costs and increase efficiency based on the pay-as-you-go model, and maximize the commercial value of technology.

In this wave of complementary technology and business, Vision Digital is a clear case. Based on Alibaba Cloud's Serverless Application Engine (SAE), Horizon Digital has been fully upgraded to a cloud-native architecture, providing the best business practice model for cloud-native applications in the entire financial technology industry.

Alibaba Cloud Ecological Fintech Industry Benchmark——

Founded in 2014, Horizon Digital is the first big data financial information service provider in China that focuses on serving the primary market, corporate credit, industrial planning and investment promotion, and facing the multi-level capital market. Different from the operation logic of other to Cs, Horizon Digital deeply binds data products to the usage scenarios of the financial industry, and has launched a series of business function modules.

With the continuous upgrading of business, Vision Digital started from providing data and information services for financial institutions. In recent years, its customers have gradually expanded to serve governments, industrial parks, large state-owned enterprise groups, etc., and the scope of data coverage has gradually extended from financial-related data to industrial products. Data, business operation data, macroeconomic data, government affairs data, policy public opinion data, geographic information data, etc., around the digital transformation of customers, to provide customers with data sales, data integration, data processing, data middle-office, system development and big data models The overall data service solution of analysis and consulting services helps the digital transformation of China's industries.

In addition to the role of financial data provider, today's Vision Digital is entering a new stage of development with multiple product lines and multiple types of service customers, becoming a true financial industry data service solution provider.

Business pain points first, prefer Alibaba Cloud Serverless solution

In the financial industry system, the application scenarios of big data have always been synonymous with sensitivity and complexity.

Taking data invocation as an example, most financial institution systems are isolated from the external network, and data service companies are required to build a data service system that is more suitable for such scenarios or embedded data functions that can meet customer needs. For customers, Horizon Digital not only provides conventional data file/data synchronization/API data service methods, but also can seamlessly embed data into the customer's internal system in the form of SDK + embedded.

On the one hand, this simplifies the customer's development cost, and on the other hand, it greatly reduces the customer's development time for the actual use of a large amount of externally purchased data.

In addition, like many financial technology companies, Horizon Digital is a hybrid deployment model of self-built IDC + public cloud. In the exploration of migrating to the cloud, Vision Digital has always been a pioneer in the industry. As early as 2014, Vision Digital built the first-generation infrastructure cloud based on Alibaba Cloud ECS server, using a combination of open source self-built + cloud products: the entire architecture is data-centric, including SaaS-based data service platforms, secure access and protection, data service layer, data processing layer, cloud security, etc.

Vision Digital Business Architecture Diagram

As a start-up company in the technology industry, at the beginning of the business, it is necessary to quickly run the business. Initially, all applications are single-stack architecture and manual deployment, but based on the shortcomings of the technology side, these architecture optimization work has been been delayed.

However, in the past two years, such problems based on technical architecture have become increasingly prominent. It can be seen that data is the core asset of an enterprise's business, and data security, stability and efficiency are the keys to serving large customers. Under the inherent model, the Horizon Digital test environment cannot obtain the full amount of real customer data, and many cases cannot be covered. It can only be exposed in the process of frequent release & testing in a grayscale environment (equivalent to pre-release) before going online. some problems:

1) Slow development iteration efficiency: single chimney architecture, high code coupling, slow development efficiency.

2) The online process is complicated and the cost is high: uses SVN code management + manual deployment, and lacks standardized DevOps processes. Before each launch, the three teams of R&D, quality inspection, and operation and maintenance need a lot of collaboration in the grayscale environment, and they go back and forth. 20 to 30 data verifications, frequent release tests, and poor development and operation and maintenance happiness.

3) Capacity estimation cannot be automated: Every time there is a marketing activity/important event on the client side (such as Xinhua Finance and financial rankings, etc.), it is necessary to inform Horizon Digital to prepare ECS one week in advance. .

In response to the above problems, Horizon Digital's own technical framework upgrade has been on the line. It is understood that two plans have been discussed within Vision Digital:

Option 1: ECS self-built Docker + open source microservices, found that it can quickly containerize and improve resource utilization, but the underlying infrastructure operation and maintenance (DockerDaemon upgrade, configuration management, image warehouse management, etc.) and development workload (microservice components self-research) , the risk of online operation and maintenance is high. After a simple POC, a collective abandonment decision is made.

Option 2: Use a commercial microservice PaaS platform to host applications, and find that it can lower the threshold for microservices and ensure the stability of microservice components. However, ECS still needs to operate and maintain itself, which is still very cumbersome, and the overall cost is too high and exceeds the budget.

Finally, I learned about SAE at a technical communication meeting, and combined with the company's technical background at the time, I found that and the company's technology upgrades have a high degree of fit. The complete experience of Microservice + Serverless + K8s is ready to use out of the box and eliminates the need for later operation and maintenance. Vision Digital has opened up the practice of architecture upgrade.

Comprehensive upgrade of technical architecture

If a worker wants to do a good job, he must first sharpen his tools. Before formally migrating the business, the first thing Horizon Digital did was to standardize the online process, hoping to reduce the burden through continuous integration.

1) Build a cloud-native DevOps system of Git + Jenkins + SAE from 0 to 1.

2) Complete the transformation of the microservice architecture through the low threshold of SAE, and upgrade to the microservice + K8s + Serverless architecture in one step.

In the early stage, a new version of the main line product, the anti-crawling identification application, was selected and tried to split the microservices. After splitting, it is developed based on Spring Cloud standards and then deployed to SAE. In the process, it was found that SAE's support for Java microservices is too complete: customers do not need to consider data isolation, distributed transactions, circuit breaker design, current limiting and downgrading, etc., and they do not need to worry about the limited community maintenance. The problem of secondary custom development, Out of the box, it greatly improves development efficiency. Moreover, on the basis of open source, SAE provides advanced service governance capabilities such as lossless online and offline, service authentication, and full-link grayscale through deep integration of MSE. Help customers shield K8s technical details, allow customers to containerize with zero threshold, and embrace K8s without feeling.

In the process of practicing SAE, the strategy of independent business + user grayscale is adopted, the traffic is gradually enlarged, part of the business is launched one after another, and then the historical stock applications are migrated.

Continue to evolve and build a financial-grade cloud platform

Due to the particularity of the financial industry, in the process of upgrading the ECS architecture to a serverless architecture, at the beginning, I was worried that SAE could not meet the regulatory requirements for financial security compliance. However, after communicating and confirming with SAE students, as well as the continuous evolution of SAE products, the concerns of vision were completely dispelled.

1) Equal protection compliance: uses cloud shield, firewall, DDOS, fortress and other security protection products in ECS mode can continue to be used, SAE also provides intrusion detection and vulnerability scanning. Later, SAE also supported the deployment of applications in the Enterprise Edition of the Container Image Service, and supported image security scanning and multi-dimensional vulnerability reporting to ensure storage and content security.

2) Security isolation: SAE students inform users that there is no influx of traffic, and that selecting JDK as Dragonwell can also support communication encryption in the future. The bottom layer is based on the security sandbox container + VPC network, which can realize multiple security isolation of system + network + data.

3) Operation audit: For some operations of on SAE, the change history can be traced back through SAE's unique release sheet. At the same time, SAE is also connected to the cloud product operation audit, and can query all operation behavior logs and addition, deletion, modification and inspection events on the cloud.

4) Permission control: SAE also solves a long-standing problem: permission isolation and approval. In the past ECS mode, especially when a new employee arrives at the post or cross-team joint adjustment, it is very cumbersome to configure user groups, RAM permissions, and login and connection methods for new machines, and account management personnel often become the bottleneck. More importantly, the operation and maintenance operations are not approved, and the risks are uncontrollable. The development has the user name and password of the machine, and the release is relatively random.

After using SAE, you can add permissions at the application granularity, and you only need to add one application per application, which saves you worry and effort. SAE also designed the operation and maintenance approval process through the main and sub-accounts, which effectively restrained the quality risks caused by random online publishing.

operation and maintenance efficiency is increased by 60%, and the effect is remarkable (click for details)

Through continuous running-in verification with the SAE platform, some applications of Vision Digital have been gradually migrated to SAE. The entire migration process is smooth, without any modification cost, zero failure, and only 1 R&D personnel has been invested. Next, we plan to fully migrate the overall architecture to SAE to fully enjoy the benefits of cloud-native technology.

To build a benchmark in the financial technology industry, enterprise big data has great potential

At present, many industries such as finance, industry, agriculture, etc. are walking on the fast lane of digitization.

The financial industry is special enough. On the one hand, its data sources are relatively standardized, on the other hand, the coverage and application of financial data are relatively extensive, and the industry has a high degree of recognition of data. Today, Vision Digital is well-known in the industry based on its experience in serving core financial institutions, and its data quality has won unanimous praise in the industry.

It is understood that the current vision digital science has created a number of product lines, not only a "scenario-free" data query platform, but also a business functional platform based on specific business scenarios. In addition to industry-standard data, it also provides deeper and more comprehensive labels for different data.

It is commendable that Horizon Digital has invested a lot of research efforts in more than 100 industrial chains of the national economy, more than 6,000 sub-sectors and nearly 100,000 categories of "products and services", and has produced relatively accurate and complete products. Industry classification system. With the deep mining of data labels, it has built its deep processing and recalculation capabilities for various types of enterprise-related data.

Summarize

In the future, the application of financial technology will become a new growth point in China's financial industry, and financial innovation driven by technology will further help finance and various industries to achieve a substantial increase in efficiency. Based on this opportunity, Vision Digital will also sing on the waves, continue to cultivate in its own track, and promote the digital and intelligent development and transformation of the financial industry with the kinetic energy of industry and finance data, and help the new pattern of China's financial technology development.

Click here to view SAE related product information!