NetEase Yunxin joins hands with "Yaotai" to create a benchmark case for meta-universe commercialization practice

In the context of the development of software and hardware technology space, people's imagination of Internet gameplay is gradually moving from creating virtual space to integrating the real world and virtual space.

Even in the past October, Facebook founder Mark Zuckerberg changed the name of "Facebook" to "Meta", which made "Meta Universe" the limelight in the entire technology circle and investment circle.

Speaking of "meta universe", according to the current development form, it is a process of copying and recreating real space, that is, people recreate a virtual world with a high sense of immersion, experience, and interaction on the Internet, and even people You can reshape your life based on the virtual world created by various Internet giants.

Of course, if we really want to achieve such an ultimate goal, there is actually a long way to go, but the outbreak of the new crown epidemic will undoubtedly provide more market-oriented demand for the commercial use cases of "meta universe".

For example, in our common meeting scenes, most of the meetings before the COVID-19 pandemic were offline meetings. When offline, participants can choose to have real-time communication with others in 1-to-1, 1-to-many, or even many-to-many at any time, or even In the meeting, you can temporarily reserve a "small meeting room" to meet your best friends, but such an experience with a strong sense of scene is not available in traditional video conferences.

At present, mainstream traditional video conferences are relatively simple in presentation mode and lack space scenes, so that the user layer lacks a sense of presence during the conference, and is prone to mental fatigue and inattention.

Focus on application experience and create a "new world" for immersive meetings

In order to solve this problem, Netease Fuxi took the lead in releasing an immersive conference activity system-"Yaotai" to facilitate people to hold various conferences online during the epidemic.

(Login interface of "

"Yaotai" itself creates a brand-new ancient immersive virtual meeting world. In "Yaotai", event parties can set up various styles of event venues in the scene according to their actual needs, and reproduce real meetings with high precision. Scenes.
At the same time, in order to enhance the interactivity and experience of the meeting, "Yaotai" not only implements dozens of meeting functions such as PPT embedded playback, text and voice chat, simultaneous translation, and free switching of multiple sub-venues, but also to satisfy In order to meet the social needs of the meeting, there is no need to add a group, and the virtual person can start a group chat when they meet.

(the virtual person approaches and turns on the "group chat mode")

Considering that the virtual conference world is closer to the real conference scene, the conference organizer can freely choose the number, layout and content of exhibition boards in the event venue in "Yaotai"; the number and layout of seats in the event venue; the venue is loaded with advertisements; large screens and exhibition boards publicity advertisements ; Event venue naming, etc. At the same time, "Yaotai" also supports functions such as back-end independent management of meetings and meeting data analysis

(tailor-made clothing for DAI 2020. Image source: Netease)

It is worth mentioning that, in order to increase the authenticity and sense of participation in the user experience, participants can create their own personal avatars by uploading personalized photos.
Relying on the AI smart face pinching technology pioneered by NetEase Fuxi, the platform will accurately restore portrait photos and facial features, analyze facial features through the portrait network, reconstruct the three-dimensional information of the face, and combine stylized network learning data to generate specific Stylized appearance, such as hairstyle, skin tone, facial features.
At the same time, the facial expressions and actions of the virtual person will change in real time as the real facial expressions and actions change, making all participants feel as if they are in a real meeting and intuitively feel each other's happiness, anger, sorrow, and joy.

(Exclusive avatar and real person driving avatar example)

At present, "Yaotai" can be applied to many scenarios such as international academic conferences, signing ceremonies, online exhibitions, job fairs, new product launches, brand promotion meetings, company annual meetings, award ceremonies, round tables, campus presentations, etc. And it can hold up to 500 people at the same time to participate in the main venue at the same time.
In addition, considering the frequency of participants participating in such activities and maximizing the experience of participants, "Yaotai" chose the simplest way, which is to log in to the platform through a browser and enter the participant code to enter the meeting scene. A series of tedious processes such as downloading the client and registering an account are eliminated.

cultivate the underlying technology to create the ultimate interactive experience of "

As a converged communication cloud service expert built by NetEase's 24 years of IM and audio and video technology, NetEase Cloud Trust provides core capabilities and components for converged communications and audio and video. In the cooperation with Netease Fuxi, "Yaotai" has deeply integrated NetEase Cloud's audio, video, and screen sharing capabilities.

As an immersive conference system, "Yaotai" not only provides users with a highly immersive and highly interactive conference activity experience, but also needs to do a good job in the audiovisual experience of the user and the sharing quality of the activity party.

1. Self-developed AI algorithm to ensure users enjoy high-quality audio-visual experience

NetEase Yunxin's self-developed AI audio and video engine supports single and dual channels, audio sampling rate 16 kHz-48 kHz, bit rate up to 128 kbps stereo high-definition sound quality; in video calls, it supports 1080P high-definition image quality. At the same time, in order to avoid possible noise problems during calls, NetEase Yunxin uses self-developed audio 3A algorithm processing and AI audio noise reduction algorithms to help users eliminate echo and howling during calls, and to deal with noises such as noisy human voices and keyboard sounds. Steady-state noise carries out directional noise reduction, so as to ensure that "Yaotai" users enjoy a pure call and interactive experience.

(Participants conduct real-time interactive exchanges on the content of the wallboard)

In addition, NetEase Yunxin audio and video technology supports users to join multiple rooms at the same time in "Yaotai", so that they can subscribe to the audio and video streams of multiple conference rooms at the same time. At the same time, they can also push the local audio and video streams to a certain one. In the meeting room, 16 people can chat at close range in the meeting scene.

Taking into account the possible network delay problems in user sharing and interaction, NetEase Yunxin relies on leading precision bandwidth detection, intelligent congestion control, forward error correction, coding optimization and other technologies, and the measured anti-packet rate reaches 80%, thereby ensuring "Yaotai" users can still enjoy a high-stability, low-latency audio-visual experience even in a complex network environment.

2. Highly available network, comprehensively guarantee the global interaction of

At present, "Yaotai" has already involved many types of international academic conferences in the application scenarios, which needs to ensure the real-time transmission quality of various data under long-distance and highly complex network conditions. Netease Yunxin released the first transport layer base independent of business logic-the global intelligent routing network WE-CAN, which perfectly supports media streaming, IM messages, signaling and general data transmission, and is 99.9% of calls made at "Yaotai" Provide non-stop audio and video services.

(WE-CAN global intelligent routing network hierarchical diagram)

Currently, WE-CAN has been deployed in 200+ major regions around the world such as India, the Middle East, Europe, North America, and North Africa. Extensive node coverage, perfect operation mechanism, flexible scheduling strategy and intelligent routing strategy, these 4 characteristics give WE-CAN "transportation gate" general attributes, no matter where the user is, as long as the data enters the system, they can quickly reach the destination .

3. Real-time sharing, creating a simple and easy-to-use sharing environment

NetEase Yunxin provides "Yaotai" with a simple and easy-to-use NERTC SDK. After simple integration, the speakers only need to share their screen content such as files, data, web pages, PPT and other screens in the way of screen sharing during the meeting. To the attending audience, so that other participants can more intuitively understand the content and topics of the discussion, and improve communication efficiency.

(Real-time meeting sharing)

In addition, taking into account the issue of access costs, Netease Yunxin helps "Yaotai" save access costs through Unity SDK packaging technology.
With the gradual popularization of virtual reality technology and audio and video call technology, high-demand, multi-scene, and low-threshold applications will promote the accelerated explosion of "Yaotai", and many "AI + conference" gameplay innovations are also continuing to evolve.

In the next step, "Yaotai" will further develop functions such as virtual communities, virtual receptions, and VR virtual reality. NetEase Yunxin will also continue to expand its service boundaries to include conference meta-universe, game meta-universe, social meta-universe, and education meta-universe. And other scenarios continue to provide stable, mobile, and reliable converged communication cloud services.

Recently, NetEase Yunxin and NetEase Fuxi Lab launched the industry's first "virtual image + RTC" integrated SDK, and based on this integrated SDK, a real-time interactive solution for Netease Yunxin virtual image is formed. This solution not only can vividly restore the image of the avatar, but also combines the real-time transmission capability of Netease Yunxin WE-CAN (Communications Acceleration Network), which can realize real-time interaction with the avatar, helping enterprise customers to achieve real-time interactive scenes of avatars with zero threshold. Take the first step towards the meta universe.

Fill out the form and try it for free now >>>

NetEase Yunxin's virtual image real-time interactive solution has six advantages:

1. Image: Highly restored, extremely agile.
2. Hardware: No need to wear a device, mobile phone can be realized.
3. Performance: End-to-cloud collaboration, thousands of yuan machine can also be played smoothly.
4. Interaction: Low latency and no lag, comparable to "face-to-face communication".
5. Convenience: 1 SDK implements 2 core technologies.
6. Scenario: Multi-industry application, reducing costs and increasing efficiency is one step faster.

Fill out the form and try it for free now >>>