Alibaba Cloud Video Cloud, using technology to create civilian "virtual people"

Recently, AYAYI, the first ultra-realistic digital person in China, announced that he has joined Alibaba and became the digital manager of Tmall Super Brand Day. The two will jointly open the marketing world of Meta Universe. In the future, she will unlock multiple identities with Tmall, such as NFT artist, digital curator, fashion brand manager, top digital person and so on.

When it comes to virtual digital people, the first time everyone thinks of it is Luo Tianyi, who was born in 2012. Her appearance fills up the blank virtual idol market in China. Although she only had sound and paintings at first, for nine years, fans have been writing songs and composing music, drawing illustrations, and making MMD for her. These fan creations filled the initial gaps little by little. Compared with real-life idols, virtual idols growing up on UGC content are more like a purely cultivated starmaking. At the beginning of 2021, the virtual idols with dimensional characteristics represented by Luo Tianyi began to shine in the Spring Festival Gala, which means that the respect and recognition of dimensional culture by the national platform is also a beneficial interaction between virtual and reality.

Luo Tianyi Appears in the Spring Festival Gala Show Children's Song and Dance "Listen to Me"

According to data from iiMedia Research, the scale of China’s virtual idol’s core industry in 2020 is 3.46 billion yuan, a year-on-year increase of 70.3%, and is expected to reach 6.22 billion yuan in 2021; on the other hand, as the commercial value is continuously discovered, more More and more industries are linked with virtual idols. The virtual idols will drive an industry scale of 64.56 billion yuan in 2020, and it is expected to be 107.49 billion yuan in 2021.
On foreign short video platforms, virtual digital people are also popular. A virtual idol named Lil Miquela is set as a 19-year-old mixed-race girl with cute freckles who has long lived in Los Angeles and has a virtual model and musician of Hispanic, Brazilian and American descent. According to a report by CCTV on January 30, 2021, Lil Miquela's revenue in 2019 is equivalent to approximately 76 million yuan.

What is a virtual digital person?

The term "virtual digital human" originated from the "Visible Human Project" (YHP) initiated by the National Library of Medicine in 1989. In 2001, the 174th academic symposium of the Xiangshan Science Conference with the theme of "Science and Technology Issues of China's Digital Virtual Human Body" put forward the concept of "digital virtual human body". These "virtual digital humans" mainly refer to the visualization of the human body structure, displaying the size, shape, position and inter-spatial relationship of the human anatomical structure in three-dimensional form, that is, using human body information to realize the digitization of the human anatomical structure. Mainly used in human anatomy teaching, clinical diagnosis and treatment in the medical field.

Different from the above-mentioned digital human body in the medical field, the virtual digital human (hereinafter referred to as "digital human") discussed around Metaverse on the Internet refers to a virtual human with a digital appearance. Unlike physical robots, virtual digital humans rely on display devices to exist. Virtual digital humans should have the following three characteristics: one is the appearance of the owner, with specific features such as appearance, gender and personality; the second is the behavior of the owner, with the ability to express with language, facial expressions and body movements; 3. It is the thought of the owner, the ability to recognize the external environment, and to communicate and interact with others.

In recent years, thanks to breakthroughs in deep learning algorithms, the production process of digital humans has become simpler. Virtual digital humans have begun to enter the public’s field of vision and are used in various industries. For example, entertainment digital humans (such as virtual anchors, Virtual idols), educational digital people (such as virtual teachers), assistant digital people (such as virtual customer service, virtual tour guide, intelligent assistant), film and television digital people (such as stuntman or virtual actor), etc. Virtual digital humans with different shapes and different functions empower film, media, games, finance, cultural tourism and other fields, and provide users with customized services according to their needs.

Virtual Digital Human Industry View Source: "White Paper on the Development of Virtual Digital Human in 2020"

In terms of implementation technology, the virtual digital human system is generally composed of five modules: character image, voice generation, animation generation, audio and video synthesis and display, and interaction. The most basic is the character image technology, which can be divided into 2D and 3D according to the dimensions of the character graphic resources. From the appearance, it can be divided into cartoon, anthropomorphic, realistic, super-realistic and other styles.

Source: "White Paper on the Development of Virtual Digital Humans in 2020"

Alibaba Cloud Cartoon: Create a virtual idol around you

Because the virtual person's character image construction has a certain technical threshold, it seems to ordinary people that the virtual person is still at the stage of idols and can only "look away". However, the team from Alibaba Cloud Video Cloud has used optimized algorithm technology to give ordinary people the opportunity to create their own cartoon images, and to create "civilian idols" in the virtual world. In the developer exhibition area of the Yunqi Conference in 2021, Alibaba Cloud demonstrated the latest technology-the cartoon intelligent drawing project, which attracted nearly 2,000 experiencers and became a hit at the conference. This technology adopts the face stylization technology scheme of latent variable mapping. Through the machine learning algorithm, the original facial features of the photographer and the target training style can be deeply integrated, and the facial shape can be reconstructed. It not only conforms to the user's facial features, but also With cartoon stylization, it can also track the user's facial posture and rich expressions to generate real-time animation.

And all of these technologies are done by AI. For the user, just upload a selfie to automatically generate a virtual person that meets the user's facial characteristics.

Alibaba Cloud Cartoons can discover the salient features (such as eye size, nose shape, etc.) of the input portrait image, and automatically generate a personally unique virtual image (that is, the stylized effect). The so-called face stylization is to convert the real face avatar into a specific style avatar. The most common ones include cartoon style, anime style, and oil painting style, as shown in the following figure:

Alibaba Cloud Video Cloud Cartoon Zhihui Effect Picture

Thanks to the rapid development of AI technology, virtual humans are no longer inaccessible. In the foreseeable future, virtual humans created by Cartoon Zhihui will serve as the digital avatars of virtual anchors, virtual teachers, virtual customer services, and virtual tour guides. , Appearing more and more frequently in all walks of life.

Reference materials: China Artificial Intelligence Industry Development Alliance General Group and Zhongguancun Digital Intelligence Industry Alliance Digital Human Working Committee, "White Paper on the Development of Virtual Digital Humans in 2020"

"Video Cloud Technology" Your most noteworthy audio and video technology public account, pushes practical technical articles from the front line of Alibaba Cloud every week, and exchanges and exchanges with first-class engineers in the audio and video field. The official account backstage reply [Technology] You can join the Alibaba Cloud Video Cloud Product Technology Exchange Group, discuss audio and video technologies with industry leaders, and get more industry latest information.