Revealing the black technology of the 2022 Winter Olympics, how does Alibaba Cloud video cloud "Cloud ME" realize holographic meeting?

The 2022 Beijing Winter Olympics was originally an extraordinary existence. For the first time in history, all the core systems required for the Olympic Games were fully integrated into the cloud, creating a new era of the Olympic Games with digital technology.

But the Green Olympics is more than that. In the face of Covid-19 restrictions and physical barriers, thousands of miles away, people-to-people meetings can be realized in a holographic form through the light of green technology. "Alibaba Cloud ME", an innovative solution on the cloud launched by Alibaba Cloud Audio and Video Communication Technology), spanned the physical space for the first time in the history of the 100-year Olympic Games, creating a "cloud gathering" moment between people. (hereinafter referred to as Cloud ME)

The warmth of the gurgling at the cloud gathering site - Cloud ME creates a legendary meeting

The original intention of "Cloud ME" is to provide an immersive interactive experience for people who are eager to explore the real sense of meeting under the constraints of physical space, so that they can meet in a 3D form of real people and enjoy real-time dialogue and interaction with each other.

Just after the opening of the Winter Olympics, Bach, the chairman of the International Olympic Committee, who was in the closed loop of Beijing, "appeared" in the cloud gathering exhibition area of the 2022 Beijing News Center through Cloud ME, and Zhang Yong, chairman and CEO of Alibaba Group, who was far away in Shanghai, "Clouds Gathering" in the air, sending the Olympic message on the same stage in a holographic form, and passing the Winter Olympics torch "on the spot" in the air. Witnessed by many media reporters, Cloud ME made all conversations and interactions as if real people were on the scene, and the atmosphere Real and fluid, marveling at the early arrival of the Metaverse, and deeply feeling the gurgling warmth of technology.

Bach and Mr. Zhang Yong, who were 1,300 kilometers apart, appeared in the Beijing News Center in a holographic form to celebrate the New Year on the same stage.

In addition to the meeting in the Olympic century, what is even more warm is that through Cloud ME, families of medical staff who have been separated from Beijing and Shanghai due to the epidemic can be "reunited with the whole family on the cloud". The image, and a family portrait that travels through space, is so warm that it is enough to instantly melt the arduous evolution of all the technologies behind it. At this moment, technology also wields the shining light of humanity.

Through Cloud ME, a special meeting

The entire meeting process uses Cloud ME to project the full-body images of the participants into the remote holographic capsule to display their 3D images, thereby facilitating multi-person remote meeting and interaction in different places.

The Cloud ME holographic capsule is simple to install and, apart from typical studio equipment (such as cameras and computers to capture footage), is based on powerful cloud computing capabilities and does not require any additional on-site IT infrastructure or optimized Internet bandwidth to facilitate recording and recording. transmission. It is the characteristics of Cloud ME that are easy to implement, so that virtual reality meetings can be opened to a wider audience in more unlimited fields, creating more diverse and warmer cloud gathering possibilities.

A special and warm "family portrait"

In this regard, no matter how many people you want to meet, how far away you are, and how difficult it is for teams to meet, you can use Alibaba Cloud Video Cloud's Cloud ME to meet each other within a short distance.

The Magical Appearance of Cloud ME - Hair-Level Holographic Reproduction & Blink-like Ultra-Low Latency

Based on Alibaba's support in digital and intelligent technologies, the audio and video produced by Cloud ME will be compressed, transmitted, processed and rendered on Alibaba Cloud. In this process, the time-honed RTC technology plays the most critical role, while Alibaba The cloud's advanced computing capabilities and extensive global network coverage also provide a solid foundation for RTC.

To realize multi-person remote holographic meeting, the operation of Cloud ME is simple, but the technology is extraordinary.

https://www.youku.com/video/XNTg0NDQ5MzcwOA==
The technical realization of the Winter Olympics Black Technology Cloud ME

The legendary process is as follows:

First of all, in the image capture process, the images captured by the camera cannot be directly imaged on the playback side. All images need to be deeply compressed by advanced encoders, and implemented in real-time transmission protocols and QoS (Quality of Service) anti-packet loss policies. It is sent to Alibaba Cloud's audio and video communication (RTC) network under the guarantee of the video, and the format and size of the final uploaded picture are no different from ordinary 4K movies.

Then, when the picture reaches the cloud, it needs to use the powerful processing capabilities of the cloud to complete the complex process including transcoding, rendering, scene reconstruction, light and shadow processing, and complete the transformation from "plane" to "holographic". : 1-scale virtual reproduction, generate 3D holographic characters, reconstruct all interactive details in high-definition, including the characters' micro-facial expressions and clothing textures, achieve "hair-level" holographic full-scale reproduction, and create a full-real interactive immersion Feeling, everything is like being there.

UI. In such a complex and large-span technical link, in order to achieve a smooth meeting effect, the key to technical realization is "speed", and the key to ensuring "speed" is Alibaba Cloud's global audio and video transmission network GRTN (Global Network). Realtime Transport Network), and its core technologies in video encoding and decoding, network transmission, ultra-large-scale audio and video processing and scheduling.

Based on these technologies, the entire process of image transmission and processing, including SDK encoding, real-time transmission, cloud processing and rendering, end-side decoding and imaging, can be completed in just 200 milliseconds in the blink of an eye. The comprehensive sense of communication and interaction is the same, there is no sense of delay, and it is extremely real.

As a result, Cloud ME's hair-level full-scale holographic engraving, blink-like ultra-low latency, and rock-solid network transmission power allow more interactive miracles to happen.

The Pratt & Whitney Light of AliRTC's Ultimate Technology - The "Zero Processing" Era Comes With The Trend

The technical realization of Cloud ME relies on the AliRTC "zero processing" solution proposed by Alibaba Cloud Video Cloud, that is, through the "cloud processing + terminal rendering" technology, the hardware side can only perform simple collection, transmission, and playback, and at the same time, All complex work is done on the cloud, allowing the terminal to achieve "zero processing".

This is the first time in the history of the 100-year Olympic Games that the holographic RTC technology on the cloud has been used to realize the technical application of cross-scenario remote meeting, which completely broke the physical space barrier during the epidemic period. Through Cloud ME, athletes, event staff, volunteers and media friends can be "teleported" to a designated location for real-time interactive communication.

In the process of experiencing Cloud ME, the host of Hunan Satellite TV's "News and Truth" column "teleported" the volunteers from the Beijing Winter Olympics to the "program site" to conduct "face-to-face" super interviews, realizing the realization of time and space. Synchronize. Xu Wenjun, a volunteer from Tsinghua University, said: "It feels like we are communicating face-to-face!".

Hunan Satellite TV's "News Seeking Truth" column conducted on-site experience and program interviews with Cloud ME

The "Cloud ME" of Alibaba Cloud Video Cloud can be used in the Olympic Games scenarios and outside the Olympic Games. It can be widely opened to more participants and fans, allowing them to experience real, immersive and interesting holographic interactions in virtual reality. The new content and new interaction of the Centennial Olympic Games are truly opened.

Not only that, based on the global deployment of Alibaba Cloud RTC, it can also realize the scene of "online in one place, synchronous interactive presentation in multiple places", and has the phenomenon-level energy of "no longer lack of avatars".

RTC audio and video communication has proven to be a key technology that helps the world connect effectively. With the help of Alibaba Cloud's global network and intelligent algorithms of Video Cloud, Alibaba Cloud Video Cloud aims to bring audio and video communication capabilities into a wider range of application scenarios, not only in interactive entertainment, smart education, video conferencing, but also more. Valuable enterprise services and more imaginative immersive interaction fields.

From graphics to audio and video, the first interactive change in society occurred in 2014, and RTC technology drove the subversive development of interactive video technology and online interactive commerce in 2017. With the development of 5G technology, it will bring a new round of interactive evolution, such as VR remote collaboration, virtual reality interaction, etc.

Among cloud vendors, Alibaba Cloud Video Cloud took the lead in proposing the "cloud processing + terminal rendering" technology. Through the combination of real-time communication and cloud processing technology, it proactively solved the problem that new interactive experiences could not be realized due to the limited computing power on the terminal side. The virtual interactive scene problem of Alibaba Cloud has driven the era of terminal "zero processing" to follow the trend, and Cloud ME is the innovative practice of Alibaba Cloud Video Cloud in this field.

In the future, AliRTC will make full use of the cloud-integrated hybrid computing capabilities to help the commercialization of more innovative scenarios, such as holographic concerts, virtual concerts, innovative cloud exhibitions, holographic interactive entertainment, etc. The evolution of the ultimate experience with more immersive interaction.

RTC is technology, technology is a tool, and all its end points are the warmth of humanistic connotation, which is the invisible force that breaks the world of Erjiang and our world, and it is the closeness between people and people, people and things, and people and the world. between.

Looking forward to Cloud ME, looking forward to AliRTC, looking forward to the future.

---

"Video Cloud Technology", your most noteworthy public account of audio and video technology, pushes practical technical articles from the frontline of Alibaba Cloud every week, where you can communicate with first-class engineers in the audio and video field. Reply to [Technology] in the background of the official account, you can join the Alibaba Cloud video cloud product technology exchange group, discuss audio and video technology with industry leaders, and obtain more latest industry information.