Good news for audio and video developers, quickly integrating AI dubbing capabilities

Audio and video content creators have an increasing demand for AI dubbing in editing applications. It has many advantages. It can not only solve the cost of hiring human dubbing and the limitations of the creator's own accent and language, but also greatly improve production efficiency. For example, the playback time of a short video is as short as tens of seconds, and a long vlog type video is 4-5 minutes. Using the AI dubbing function, multiple editing needs can be achieved in a short time.

HMS Core audio editing service (Audio Editor Kit) provides AI dubbing service to help developers easily build speech synthesis functions in applications, and output text with one click to convert speech. It supports a variety of styles such as cute children's voices, friendly female voices, sunny male voices, English male voices, female voices, etc. You can also customize the timbre you want. The timbre is natural and smooth, and the emotion is rich, which meets the popular scenes that developers pay attention to, such as audio reading, audio content production, and editing.

development practice

1. Development preparation

For detailed preparation steps, please refer to the official website of Huawei Developer Alliance:

https://developer.huawei.com/consumer/cn/doc/development/Media-Guides/config-agc-0000001154009063?ha_source=hms1

2. Edit engineering integration

2.1 Set the authentication information of the application

Developers need to set application authentication information through api_key or Access Token.

• (Recommended) Set the Access Token through the setAccessToken method, and initialize the settings when the application starts

HAEApplication.getInstance().setAccessToken("your access token");

• The api_key is set by the setApiKey method, which can be initialized and set once when the application starts, and there is no need to set it multiple times.

  HAEApplication.getInstance().setApiKey("your ApiKey");

2.2 Initialize the environment

Initialize the audio editing management class, create the timeline, and the required swimlanes.

// 创建音频编辑管理类
HuaweiAudioEditor mEditor = HuaweiAudioEditor.create(mContext);
// 初始化Editor的运行环境
mEditor.initEnvironment();
// 创建时间线
HAETimeLine mTimeLine = mEditor.getTimeLine();
// 创建泳道
HAEAudioLane audioLane = mTimeLine.appendAudioLane();

Import music.

// 泳道末尾添加音频资源
HAEAudioAsset audioAsset = audioLane.appendAudioAsset("/sdcard/download/test.mp3", mTimeLine.getCurrentTime());

3. AI dubbing function integration

Call HAEAiDubbingEngine to implement AI dubbing function.

// 通过此配置类来对AI配音引擎进行配置
HAEAiDubbingConfig haeAiDubbingConfig = new HAEAiDubbingConfig()
// 设置音量
.setVolume(volumeVal)
// 设置音速
.setSpeed(speedVal)
// 设置发音人
.setType(defaultSpeakerType);
// 单个AI配音任务回调
HAEAiDubbingCallback callback = new HAEAiDubbingCallback() {
    @Override
    public void onError(String taskId, HAEAiDubbingError err) {
        // error处理
    }
    @Override
    public void onWarn(String taskId, HAEAiDubbingWarn warn) {}
    @Override
    public void onRangeStart(String taskId, int start, int end) {}
    @Override
    public void onAudioAvailable(String taskId, HAEAiDubbingAudioInfo haeAiDubbingAudioFragment, int i, Pair<Integer, Integer> pair, Bundle bundle) {
        // 开始接收文件，保存成文件
    }
    @Override
    public void onEvent(String taskId, int eventID, Bundle bundle) {
        // The synthesis is complete.
        if (eventID == HAEAiDubbingConstants.EVENT_SYNTHESIS_COMPLETE) {
            // AI配音任务处理结束，即合成的音频数据全部处理完毕
        }
    }
    @Override
    public void onSpeakerUpdate(List<HAEAiDubbingSpeaker> speakerList, List<String> lanList,
         List<String> lanDescList) { }
};
// AI配音引擎
HAEAiDubbingEngine mHAEAiDubbingEngine = new HAEAiDubbingEngine(haeAiDubbingConfig);
// 设置AI配音任务播放过程侦听
mHAEAiDubbingEngine.setAiDubbingCallback(callback);
// 实时AI配音并播放API，text传入待转语音的文本，modeAI配音任务播放模式
String taskId = mHAEAiDubbingEngine.speak(text, mode);
// 暂停播放
mHAEAiDubbingEngine.pause();
// 恢复播放
mHAEAiDubbingEngine.resume();
// 关闭合成
mHAEAiDubbingEngine.stop();

Demo

<Nice to meet you>

For more details about Huawei Audio Editing Service, please refer to:
Service official website: https://developer.huawei.com/consumer/cn/hms/huawei-audio-editor/?ha_source=hms1
Get guidance document: https://developer.huawei.com/consumer/cn/doc/development/Media-Guides/client-dev-0000001107465102?ha_source=hms1

Learn more >>

Visit Developer Alliance official website
Get development guidance document
Huawei Mobile Services open source warehouse address: GitHub , Gitee

Follow us and know the latest technical information of HMS Core for the first time~