The first batch + only! Alibaba Cloud's real-time computing Flink version passed the big data product stability test of the Academy of Information and Communications Technology

summary

On December 13, 2021, the results of a new round of "Big Data Product Capability Evaluation" announced by the China Academy of Information and Communications Technology. Alibaba Cloud's real-time computing Flink version passed the special evaluation of distributed stream processing platform stability, and Alibaba Cloud became the "first batch + only one". "Vendors who have passed this special stability evaluation. Previously, the Flink version of real-time computing passed the "Distributed Stream Processing Platform-Basic Capability Evaluation" and "Distributed Stream Processing Platform-Special Performance Evaluation" of the China Academy of Information and Communications Technology in May 2019 and June 2021, respectively. Therefore, Alibaba Cloud real-time computing The Flink version has become the only distributed stream processing platform product in China that has fully passed the three evaluations of basic capability, performance and stability of the China Academy

Evaluation details

In recent years, as the first authoritative evaluation system for big data products in China, "China Academy of Information and Communications Technology" has become a benchmark for product development and user procurement selection. Cloud's real-time computing Flink version, 161f2894ad17ec passed the high CPU load, high hard disk load, high memory load, high read and write load, network packet loss, single node failure, random high-intensity comprehensive tests, etc. 7 sub-projects review . Among them, the job automatically adapts to the environment under five conditions: high CPU load, high hard disk load, high memory load, high read and write load, and network packet loss, so as to run smoothly; in single node failure and random mixed tests, jobs are automatically completed from the failure. resume, and continue to run until the computing task completes. In the face of small-scale hardware failures and complex network environments, the Flink version of real-time computing exhibits extremely high architectural robustness and continues to provide users with stable enterprise-level services.

Technical advantages

The Flink version of real-time computing products has been widely used in various promotion environments of Alibaba Group, and has shown excellent capabilities in major e-commerce events such as the Tmall Double Eleven Shopping Festival and 618. this year's Double Eleven shopping festival, the total computing power of Flink's real-time computing power reached an astonishing 6.9 billion TPS per second, and the GMV large-screen delay was within 3 seconds. Real-time computing Flink version in 2021 has become the only product in China to enter Forrester's strong performer quadrant, setting a new record for domestic companies in the field of data stream computing.

Alibaba Cloud Stream Computing Products Enter Forrester Data Stream Analysis Report

Product advantages

As one of the earliest companies in China to deploy real-time computing technology, as early as 2016, Alibaba has begun to use real-time computing products on a large scale. The Alibaba Cloud real-time computing Flink version that passed the test this time is a cloud product composed of the Alibaba Cloud Flink enterprise-level development and operation platform and an enhanced kernel engine. real-time computing Flink version has more advantageous functions and stability than the open source Apache Flink. It can be used out of the box. At the same time, the enterprise-level enhancements integrate Alibaba Cloud's enhanced Iceberg, Hudi and other data lake format capabilities to facilitate users to build large-scale data lake.

Flink version also has a real-time calculation a wealth of job runs monitoring and diagnostics tuning capability , the state can provide compatibility checking and data migration status for the user, with the greatest possible reuse original state data to help users to quickly upgrade SQL job. At the same time, it supports refined resource configuration (CPU/Mem) at the operator level, and the resource utilization rate of large-scale operations is increased by 100%+.

At the product form level, the real-time computing Flink version provides users with flexible product forms to choose from. It not only supports a fully managed form suitable for users out of the box to help users quickly implement the development and operation of stream computing, but also supports a semi-managed form (based on Yarn resource allocation and scheduling) that is self-controllable and easy to integrate by users. Consistent with the user's self-built cost, it helps users realize one-stop real-time data stream processing infrastructure (including Kafka and enterprise version Flink engine VVR). For relatively small changes in resources, you can choose a monthly subscription. For resources that are difficult to clearly evaluate or have business peaks and peaks, you can choose the pay-as-you-go + Autopilot model to achieve serverless "buy as much as you spend".

The development history of real-time computing Flink products in Alibaba

Customer case

At present, Alibaba Cloud's real-time computing Flink version has accumulated nearly 100 industry case scenarios such as Internet entertainment, online transactions, finance, and online education, and multi-industry all-round practice scenarios, escorting the stable development of the business.

Real-time computing Flink version of some customer cases