Insights Live Review - Sign Language Service, Helping Communication Barrier

The ninth live broadcast of HMS Core Insights – sign language service, helping communication barrier-free, was successfully concluded on December 29th. In this live broadcast, we learned about the highlights, features, underlying technologies and evolution planning of HMS Core sign language service with our friends. Let's review the exciting content of this live broadcast together!

【Content Review】

1. Highlight features - Chinese sign language generation and avatar display.

2. Get to the bottom of things - technical details that support the realization of sign language service functions.

3. Step by step - the evolution planning of sign language services in terms of languages, thesaurus and sign language recognition.

【expert's point】

Andy, Huawei AI Algorithm Expert

Huawei started the exploration of barrier-free communication a long time ago. After continuous accumulation and exploration, we have opened up the capabilities of the HMS Core sign language service, so that more developers can work together to build a barrier-free communication environment. It can also continuously improve various functions of the sign language service in the continuous application and feedback.

Jason, Huawei Computer Vision Algorithm Expert

In HMS Core sign language service, the clear and accurate expression of sign language actions is the core. Based on the industry-leading deep neural network technology, we translate the input text into sign language movement data with stable time sequence, coherence, and the expression habits of sign language users among the hearing-impaired people in real time. Support the generation of 10 kinds of facial expressions, and strive to express accurately.

Zheng Xuan, Consultant of Sign Language Linguistics

The construction of a barrier-free communication environment has become a social trend. The HMS Core sign language service provides a technical foundation for the construction of a barrier-free communication environment through the application of advanced technology, and also enables more developers to use technology to help the hearing impaired. Sign language users in the crowd live a better and more convenient life.

【Wonderful Q&A】

Q1: What are the highlights of the sign language service?

A: The HMS Core sign language service is a sign language generation capability provided by Huawei for developers. It can translate text in real time into sign language motion data that is fluent, natural, and conforms to the expression habits of hearing-impaired people. In addition, the sign language service can clearly and accurately express sign language movements including hand movements, body gestures and facial expressions through digital human images.

Q2: What technologies are the functions of the sign language service based on?

A: The functional realization of the sign language service is mainly composed of a sign language transcription module based on machine translation and a sign language action expression generation module based on multimodal fusion. Through self-developed Chinese pre-training language model, overall replacement mechanism and acceleration technology, the accuracy and speed of sign language transcription has been improved as a whole; through multi-modal action expression fusion generation technology, sign language service can realize natural and coherent sign language action generation, in high quality In the case of less motion capture data, the accuracy and smoothness of generated actions are improved.

Q3: In which scenarios can developers use sign language services?

A: The sign language service covers daily life vocabulary, satisfying application scenarios such as chatting with family and friends, dining, shopping, etc., to achieve barrier-free communication; sign language service also fully follows the grammar rules listed in the "National General Sign Language Dictionary", not only " There are 8214 common words in the National Universal Sign Language Dictionary, as well as many high-frequency words, which help popularize the national universal sign language standard, help hearing-impaired children learn to read, and realize sign language teaching. In addition, sign language services can also enrich the social information acquisition of hearing-impaired people. Ways to shorten the distance between hearing-impaired people and society.

Q4: What is the future evolution plan of sign language service?

A: In view of the existing capabilities, the sign language service will further expand the vocabulary coverage to meet more application scenarios. At the same time, it will also enrich the expressions and gestures of the digital human, making the sign language animation more comprehensive and accurate. The sign language service will further open up new functions to create a barrier-free communication environment for more hearing-impaired people.

【learn more】

If you want to review more live broadcast moments, please log in to Huawei Developer Academy to view the live broadcast replay

If you are interested in sign language services, please log in to the sign language service homepage for more technical details.

Learn more >>

Visit Huawei Developer Alliance official website
Get the development guidance document
Huawei Mobile Services open source warehouse address: GitHub , Gitee

Follow us and know the latest technical information of HMS Core for the first time~