HMS Core machine learning service realizes simultaneous interpretation, supports Chinese-English translation and multi-timbre voice broadcast

When the user needs cross-language communication or audio content translation, the application needs to be able to automatically detect the voice content and output it into the language and text required by the user.

The HMS Core machine learning service provides simultaneous interpretation capabilities. Simultaneous interpretation realizes real-time translation of real-time input long speech into text and speech in different languages, and real-time output of original speech text, translated text, and voice broadcast of the translated text.

In live broadcast and conference applications, simultaneous interpretation is particularly important. For example, in a conference application, the speech content of the ongoing conference speaker can be output as the target language in real time, so that people in different languages can understand the conference content in real time and improve the efficiency of the conference. In live broadcast applications, the speech content of the host can be output in real time into the language and text required by the user, and then broadcast in real time, which improves the user's viewing experience.

The simultaneous interpretation capability of HMS Core's machine learning service not only supports the conversion of 5 hours of continuous real-time long speech input into text, but also supports Chinese-English translation and multi-timbral voice broadcast to meet the needs of different scenarios in different types of applications.

DEMO example

Development steps

Before developing, you need to complete the necessary development preparations, and make sure that the Maven warehouse address of the HMS Core SDK has been configured in your project, and the SDK integration of this service has been completed.

The SDK sample code is as follows:

 dependencies{
    // 引入同声传译服务SDK
implementation 'com.huawei.hms:ml-computer-si-sdk:3.5.0.312'
}

1. Please refer to the instructions for using cloud authentication information to set the authentication information of your application.
2. Create a simultaneous interpretation recognizer and set the configuration.

 MLSimultaneousInterpretationConfig config = new 
MLSimultaneousInterpretationConfig.Factory()
                // 设置源语种，目前支持中文和英文
.setSourceLanguage(MLSimultaneousInterpretationConstants.LAN_ZH_CN)
                // 设置目标语种，目前支持中文和英文
                .setTargetLanguage(MLSimultaneousInterpretationConstants.LAN_EN_US)
                // 设置识别类型。支持原始文本+翻译文本+翻译语音
                .setResultType(MLSimultaneousInterpretationConstants.RESULT_TYPE_TEXT_TRANSLATED_VOICE)
                // 设置发音人，支持中、英男声和女声
                .setPerson(MLSimultaneousInterpretationConstants.EN_FEMALE_VOICE)
                .create();
 MLSimultaneousInterpretationRecognizer = MLSimultaneousInterpretationRecognizer.getInstance();

3. Create a voice recognition result listener callback.

 // 回调实现MLSimultaneousInterpretationListener，实现接口中的方法。
    protected class SIRecognitionListener implements MLSimultaneousInterpretationListener {
        @Override
        public void onStartListening() {
            // 录音器开始接收声音。
 }
        @Override
        public void onVoiceDataReceived(byte[] data) {
            // 返回给用户原始的PCM音频流和音频能量，该接口并非运行在主线程中，返回结果需要在子线程中处理。
        }
        @Override
        public void onRecognizingResults(Bundle partialResults) {
            // 从SIRecognitionListener接收到持续语音识别的文本。
            // RESULTS_RECOGNIZING:原始文本；RESULTS_TRANS_RECOGNIZING：tts文本；RESULTS_SN：原始序列号；RESULTS_TRANS_SN：tts序列号
        }
        @Override
        public void onResults(Bundle results) {
            // 从SIRecognitionListener接收到完整的句子
            // RESULTS_RECOGNIZED:原始结果；RESULTS_TRANS_RECOGNIZED：tts结果
        }
        @Override
        public void onError(int error, String errorMessage) {
            // 识别发生错误后调用该接口。
        }
        @Override
        public void onState(int state, Bundle params) {
            // 通知应用状态发生改变。
        }
    }

4. Bind the simultaneous interpretation recognizer.

 mlSimultaneousInterpretationRecognizer.setMLSimultaneousInterpretationListener(new SIRecognitionListener());

5. Call startRecognizing to start speech recognition.

 // 开始识别。 
 mlSimultaneousInterpretationRecognizer.startRecognizing(config);

6. After the identification is completed, end the identification and release the resources.

 if (mlSimultaneousInterpretationRecognizer!= null) {
  mlSimultaneousInterpretationRecognizer.destroy();
  }

Learn more details>>

Visit the official website of Huawei Developer Alliance
Get development guidance documents
Huawei Mobile Services Open Source Warehouse Address: GitHub , Gitee

Follow us to know the latest technical information of HMS Core for the first time~