Technology evolution, scenario practice and academic exploration of Alibaba Cloud edge computing and edge cloud

With the advent of the 5G era, the Internet of Everything has entered the public life, which has put forward new requirements for the computing structure. As the computing power of the terminal moves up and the computing power of the cloud sinks, the integration of computing power is formed at the edge, and edge computing gradually penetrates into various application scenarios, becoming an indispensable network infrastructure and an important driving force supporting the high-quality development of the digital economy.

Fu Zhe, a postdoctoral researcher and technical expert of Alibaba Cloud, shared Alibaba Cloud's technological evolution route, business scenario practice and academic exploration in edge computing and edge cloud with the theme of "Edge Cloud Technology Innovation Makes "Cloud" Everywhere.

The way of data production and consumption has changed dramatically, and the development value of edge computing is highlighted

With the development of communication technology, the main body of communication has gradually migrated from people-centered to material-centered, and the flow of information has also brought about a great change in the way of production and consumption of data. The production and consumption mode of data has changed from centralized production and decentralized consumption to decentralized production and ubiquitous consumption, which means that technical reconstruction and industrial coordination are also required.

In recent years, the combination of cloud computing and 5G technology has spawned a large number of new applications and scenarios that require massive traffic, ultra-low latency, and massive connections, such as 4K/8K ultra-high-definition video, industrial control and Internet of Vehicles, environmental monitoring, Smart home, etc. However, the traditional centralized cloud model has gradually been difficult to meet the needs of these applications in terms of network bandwidth traffic, network transmission delay, and connection scale.

Edge Computing Development Trends

In the current context, in order to meet the needs of 5G applications for enhanced mobile broadband, massive terminal interconnection, and highly reliable and low-latency connections, the value of edge computing and edge cloud has become increasingly prominent.

Some institutional reports predict that in the 5G era, 80% of data and computing will take place at the edge. By converging traffic at the edge, edge cloud can realize localized processing and distribution of large traffic, avoid the impact of massive traffic on the backbone network, and effectively reduce the cost of long-distance traffic transmission.

At the same time, relying on the distributed architecture, the edge cloud can realize distributed processing of massive terminals with high concurrency, effectively improving computing efficiency. In addition, the edge cloud can also meet the scenario-based requirements for low-latency processing of massive terminals through nearby deployment.

Interpretation of edge cloud technology architecture to expand the boundaries of cloud service capabilities

Compared to central cloud or IoT, edge cloud is a new concept.

According to the interpretation of Gartner, a well-known information technology research and analysis company, edge computing is a computing method that deploys workloads at the edge compared to traditional centralized general-purpose computing. where the computation and storage are performed, and only the necessary results are sent to the cloud center.

Edge cloud and traditional cloud or IoT are complementary positioning, and there is no mutual substitution relationship. Edge cloud can be regarded as an extension of cloud, providing customers with low-latency, localization, autonomy, security and privacy service capabilities.

From the user's terminal to the cloud, Gartner divides the middle part into two types of edges:

Gartner: Edge Hierarchy

• The first is Near Edge, which is usually a non-standard server or device, where it is closest to the end side, such as inside a factory, including various types of devices such as ARM, X86, etc.
• The other type is Far Edge, which is usually a standard IDC, or MEC, such as a traditional CDN node and so on.

Both types of edge can be included in the generalized concept of edge cloud. Proximity, distribution, scenario and differentiation are the keywords that differentiate edge cloud from central cloud.

In terms of cloud computing infrastructure services, Alibaba Cloud provides a cloud computing architecture with one cloud, multiple cores, and one cloud and multiple states based on the unified Feitian base. It radiates from the center to the edge, making computing power ubiquitous.

Definition and shape of edge cloud[1]

• The central region is usually located in the first-line core area. As a large-scale public cloud form for all products, it can cope with various general elastic, high-density, large-concurrency, and high-availability scenarios, such as familiar Internet computing scenarios, big data, AI model training, high performance computing and other scenarios. The central region is usually far away from the end user, and the delay is generally within 100 ms.
• The IoT field computing nodes of the Internet of Things are located in the user's computer room and the business site, and are closest to the user, providing a computing solution that integrates software and hardware, with a delay of less than 5 ms.
• The local region between the center and the field, as well as the edge cloud node, the delay from them to the user is usually between 5 ms and 20 ms. The difference between the two is that the local region is located in an active area of the digital economy. It provides larger-scale computing services than edge cloud nodes in the form of miniaturized output from the central cloud, focusing on supporting the digital transformation scenarios of enterprises in these regions.

The edge cloud is a distributed cloud composed of large-scale geographically dispersed edge nodes, which can be remotely managed and controlled, is safe and reliable, and is standard and easy to use. [1] The single node of the edge cloud is small in scale. In the order of magnitude, the node widely covers the hotspot area closer to the user, and supports edge device management, intelligent terminal cloud, view streaming, rendering, CDN, and 5G + edge In edge scenarios such as cloud-network integration, it provides users with closer, lower-latency cloud services that are consistent with the central experience.

As one of the earliest manufacturers to define and develop edge cloud in China, Alibaba Cloud and China Electronics Standardization Institute published the industry's first "Edge Cloud Computing Technology and Standardization White Paper"[1] as early as 2018. and application scenarios are clearly defined.

Typical application coordinates of edge cloud

Latency and bandwidth, as the two most valuable advantages of edge cloud, provide the foundation for innovative application scenarios in all walks of life.

According to the requirements for latency and bandwidth, typical applications of edge cloud are listed in the graph above. In the early days, applications such as video surveillance, smart factories, VR, and cloud games were already running on the edge cloud. With the development of edge cloud technology and applications themselves, the medium and long-term edge cloud will also support innovative applications such as smart transportation, autonomous driving, and telemedicine.

Looking forward to the evolution trend of edge cloud technology, exploring innovative application scenarios of cloud services

Edge Node Service ENS

The edge node service ENS is an IaaS layer service based on operator edge nodes and networks. It provides "convergence, openness, linkage, and elasticity" of distributed computing resources, including virtual machines, bare metal, containers and other forms. Effectively help user services sink to the edge of the operator side, reducing computing delay and cost.

At present, ENS has 2,800+ nodes in China, achieving full coverage of the three major operators in 31 provinces in mainland China, and remote areas can also be accessed nearby.

At the same time, ENS provides minute-level delivery of distributed resources across the country, and users can pay by volume and expand and shrink flexibly. Relying on the advantages of edge cloud, ENS also provides a high-quality edge network, and can support edge-to-edge acceleration and cloud-edge acceleration.

In addition, ENS also provides the overall delivery capabilities of various business scenarios, such as providing mature solutions for content distribution, video migration to the cloud, etc., which help customers transform and upgrade their businesses.

View Computing VEC

Relying on the edge cloud base, Alibaba Cloud provides view computing services. View computing is for view devices, such as cameras, vehicle terminals, consumer electronics, etc. It provides PaaS services for connection, AI computing, and cloud storage for cloud scenarios of these devices, which can greatly reduce network latency and improve view data processing efficiency. .

Building VEC system architecture based on edge cloud

Based on the view computing service and Alibaba Cloud's self-developed access protocol, customers' view devices can be connected to the cloud with one click, and connected to the nearest edge node to realize basic video processing capabilities such as live broadcast, recording, screenshots, and transcoding.

At the same time, Alibaba Cloud also integrates more than 170 rich visual AI operator capabilities of Alibaba Dharma Academy, including traffic congestion, safety helmet monitoring, etc., and supports high-speed cloud access, smart construction sites and other scenarios.

The view computing service is based on the edge cloud, which can realize the processing and storage of video streams near the edge, and can optimize the traffic and storage costs for customers. In addition, the platform also provides visual process orchestration capabilities to provide users with an easy-to-use experience.

Collaborative storage of EOS

For edge large-capacity storage scenarios, Alibaba Cloud has launched an independent service - Edge Collaborative Storage.

As mentioned above, the cloud scenarios of terminals often have the characteristics of scattered locations, large data scale, and low value density. At the same time, another point is that the bandwidth is reversed, and the uplink bandwidth is much larger than the downlink. Long-term data return to the cloud will cause greater bandwidth pressure and storage costs. At the same time, the most important point is that proximity and low latency cannot be guaranteed.

Edge collaborative storage is the unified management and scheduling of object storage resources of multiple distributed nodes in the edge cloud, providing object storage capabilities with no sense of location, consistent experience, large capacity, and high cost performance.

Building EOS system architecture based on edge cloud

In order to achieve these advantages, in the architectural design, the edge collaborative storage adopts a typical cloud-edge collaborative management and control scheme. The central metadata logical bucket is dynamically mapped with the edge physical bucket to ensure data consistency. At the same time, optimized read and write scheduling strategies and algorithms are used to achieve optimal use of resources under the premise of ensuring performance and stability. In addition, edge nodes implement a part of autonomous management and control, which can further reduce access delay and improve service stability.

Global Real-time Transport Network GRTN

The third typical application is an ultra-low-latency, fully-distributed down communication-grade streaming media transmission network GRTN constructed based on central cloud and edge cloud nodes.

Traditional streaming media transmission relies on a tree-like network built on CDN.

From the image captured by a camera to the viewing by the user through the mobile phone, it needs to pass through L1, L2, live broadcast center, L2, L1 and other multi-level nodes. The link is relatively fixed, and the delay, cost, and scalability are greatly optimized. Space.

GRTN Design Overview[2]

GRTN adopts a dynamic network combining tree-like and peer-to-peer networks. The nodes of GRTN no longer have a hierarchical relationship, but are equal to each other, and finally form a system with a mesh structure.

In addition, the streaming media brain, as the core component of GRTN, is responsible for path detection, path calculation, streaming media orchestration, etc. The routing center periodically collects the results of internal link detection, and uses the KSP algorithm to perform topology calculation.

On the other hand, the link detection data between nodes cannot completely determine the actual optimal path. For example, in the scenario of a multi-person video conference, the number, distribution and even sequence of the participants will affect the final path decision. .

Therefore, the streaming media brain also needs to perceive the specific information of streaming media, and at the same time, it also needs to combine the capacity planning, cost, quality and other factors of each node to jointly arrange the optimal transmission path.

GRTN optimization results display [2]

Through tests based on actual online services, compared with the traditional CDN tree structure, the transmission delay of GRTN is increased from about 400ms to about 180ms, and the delay is halved.

In addition, in terms of user experience, 98% of the playback will not be stagnant, and 95% of the playback can start within 1s. The related results of this work have been accepted by SIGCOMM2022, and interested readers can refer to the paper further. [2]

Edge AI Utilization

Edge nodes are widely distributed and are close to where the data source is generated. They can perform specific data processing and identification optimization, and provide low-latency, low-bandwidth, low-power, and secure AI services.

Building edge AI system architecture based on edge cloud

The overall architecture adopts the cloud-edge-device three-layer collaboration solution.

On the terminal side, devices such as mobile phones and IoT devices have weak computing power and limited power consumption. Therefore, it is difficult to run complex AI models, and it is more suitable for data collection, compression, and pre-filtering.

Edge cloud provides hardware acceleration capabilities such as GPU and FPGA. However, compared with the central cloud, the scale and computing power are relatively limited, so it is not suitable for large-scale model training and persistent data storage in the edge cloud. It is suitable for inference parts with high latency requirements. Model training and persistent storage of results can be performed on the central cloud.

Therefore, in the edge AI scenario where the cloud-edge-device is coordinated, by moving AI operators from the terminal device to the edge cloud, the AI computing power is lowered from the center to the edge cloud to jointly provide low latency and high performance. AI service.

In addition, in order to facilitate algorithm scientists to deploy operator models to edge clouds, Alibaba Cloud has also developed an edge operator hosting platform, which can combine the distributed characteristics of edge clouds to provide one-stop and automated AI services on edge nodes. The deployment of the AI model transforms the reasoning process of the AI model into a common Restful API interface for end users to call.

Alibaba Cloud's experiments in a paper released at the IEEE EDGE conference in 2020 show that edge AI can greatly improve inference performance in some scenarios, and target detection performance can be improved by up to 50 times. [3]

cloud rendering

Cloud rendering, or cloud gaming, is a very hot direction recently.

Based on the fully distributed heterogeneous computing resources and network bandwidth resources of the edge cloud, Alibaba Cloud provides nearby, low-latency, location-insensitive cloud rendering services for video rendering scenarios such as games and AR/VR.

Cloud game system architecture based on edge cloud

Taking cloud games as an example, the user's game terminal only includes the display part and the operation part. The user sends control instructions to the edge cloud node. After the edge cloud node renders the real-time game screen, the video stream and audio stream of the game are sent back to the user. game terminal.

In this way, users do not need powerful game equipment, and can play the latest and hottest games through mobile phones, TVs, and even smart speakers at home.

Display of cloud game service optimization results based on edge cloud [4]

In cloud games, latency is the factor that most affects user experience. Compared with the central cloud, the edge cloud can provide network capabilities with lower latency, better quality, and lower cost. Therefore, the latency of cloud gaming services based on edge clouds is significantly better than that of cloud gaming services based on central clouds. .

Alibaba Cloud published a paper in cooperation with several universities at IMC 2021, measuring the performance and advantages of typical edge cloud applications represented by cloud games. You can learn the detailed results of the research through this paper. [4]

Directly attack edge cloud research challenges and accurately grasp the direction of business optimization

Edge-cloud collaboration challenges resource scheduling, especially resource scheduling in cloud computing, which is already a relatively mature field with rich research results. However, the birth of edge cloud brings new problems and new opportunities to this mature scenario.

In edge cloud, collaboration is an important concept.

The following three types of coordination related to scheduling will be introduced:

The first is the coordination of geographic location.

Traditional cloud resource scheduling is often a single huge cloud data center, and generally does not pay attention to the distance from the end user and the network delay when scheduling. In the resource scheduling of edge cloud, the scale of a single node is smaller, usually there are only a few hundred or even fewer servers.

However, the number of edge cloud nodes or edge cloud computer rooms is very large. For example, Alibaba Cloud has 2,800 nodes in China. In comparison, there are only about a dozen or twenty regions in the central cloud.

Therefore, in the face of this resource scheduling scenario with distributed and wide coverage of small nodes and a sense of geographic location, it is difficult for traditional cloud resource scheduling methods to achieve better scheduling results, and it is necessary to study more suitable scheduling methods for this scenario.

Second, edge cloud considers more resource dimensions.

In addition to the delay, due to the small size of a single node, it is also necessary to consider the disk size, network bandwidth, and even the number of IP addresses of nodes, the carrying capacity of the NAT gateway, and so on when scheduling.

These resources of different dimensions may be mutually dependent or even mutually exclusive. Therefore, how to achieve and do a good job of coordinated scheduling of multi-dimensional resources is also one of the challenges faced by edge clouds.

The third is the coordinated scheduling of product forms.

In traditional cloud resource scheduling, the underlying resources that products of different forms such as virtual machines, containers, and functions depend on are divided into pools, and their scheduling does not affect each other.

However, in the edge cloud scenario, it is unified fusion scheduling, that is to say, the virtual machine of customer A, the container of customer B, and the function service of customer C may run simultaneously on one server.

Therefore, how to fully improve resource utilization through scheduling on the premise of ensuring that performance does not affect each other and combining the characteristics of edge cloud products is also a big challenge.

Heterogeneous resource management

The second big research challenge comes from the management of heterogeneous resources.

For the cloud rendering cloud game scene introduced above, part of the business is currently carried by new heterogeneous hardware.

For example, ARM servers, or array servers composed of mobile phone ARM chips, and so on. For cloud computing vendors, these new types of hardware lack a standardized set of management, testing, and evaluation criteria. Alibaba Cloud looks forward to cooperating with major universities and research institutes to jointly build a set of evaluation systems and standards for edge cloud heterogeneous hardware.

In addition, based on these new heterogeneous hardware, adaptation at the virtualization level is also required. For example, how to build a fully functional container platform on a mobile phone ARM chip array server to provide more flexible and scalable service capabilities.

Furthermore, some heterogeneous hardware usually contains dedicated hardware acceleration units. Whether these hardware acceleration units can be more fully used by upper-layer services through the co-optimization of software and hardware to accelerate scenarios such as encoding and decoding, AI, etc. is also a matter of Alibaba Cloud. One of the research directions of interest.

Cloud gaming/VR latency optimization

Finally, Alibaba Cloud also has a lot of research opportunities in the recent popular application services such as cloud games, VR/AR, and Metaverse.

For example, encoding, decoding, and transmission for cloud games or VR-related scenarios can be specially optimized by integrating edge cloud features.

In addition, most cloud games currently run games directly on the edge, and only achieve "game cloudification". Although rich cloud game services have been realized in the short term, the advantages of the cloud have not been fully utilized.

In the future, whether it will be possible to create real games that run natively on the cloud. These games are born for the cloud, are flexible, and can make full use of the advantages of the cloud to bring users a more extreme cloud gaming experience. Discuss and answer questions together.

references

[1] "Edge Cloud Computing Technology and Standardization White Paper" 2018, Alibaba Cloud Computing Co., Ltd., China Electronics Standardization Institute
[2] Li, J, et al. "LiveNet: A Low-Latency Video Transport Network for Large-Scale Live Streaming." ACM SIGCOMM (2022).
[3] Fu, Zhe, et al. "Astraea: Deploy AI Services at the Edge in Elegant Ways." 2020 IEEE International Conference on Edge Computing (EDGE). IEEE, 2020.
[4] Xu M, Fu Z, Ma X, et al. From cloud to edge: a first look at public edge platforms[C]//Proceedings of the 21st ACM Internet Measurement Conference. 2021: 37-53.

"Video Cloud Technology", your most noteworthy public account of audio and video technology, pushes practical technical articles from the frontline of Alibaba Cloud every week, where you can communicate with first-class engineers in the audio and video field. Reply to [Technology] in the background of the official account, you can join the Alibaba Cloud video cloud product technology exchange group, discuss audio and video technology with industry leaders, and obtain more latest industry information.