One-click dynamization of static models, teaching you how to integrate motion capture capabilities

the interesting static model dynamization work?

Huawei 3D modeling service (3D Modeling Kit) motion capture capability, using human body detection technology, model acceleration and compression technology, and deep learning-based monocular pose estimation algorithm, only the RGB camera of ordinary mobile phones can capture the human body 24 3D information of skeleton key points, easily realize the dynamic of static model, make our model more vivid.

It is also very simple to implement this function, let's take a look at the integration steps!

Application scenarios

This service is widely used in the field of 3D content production, especially in games, film and television, medical and other industries. For example, character driving and animation video production in UGC games, real-time driving of virtual anchors, rehabilitation guidance in the medical industry, etc.

integration code

1 Development preparation

For detailed preparation steps, please refer to the official website of Huawei Developer Alliance:

Configuring AppGallery Connect-Development Preparation-Android-3D Modeling Service(huawei.com)

2 Editing project integration

Before starting API development, you need to configure AppGallery Connect in 3.3.1, and make sure that the Maven warehouse address of HMS Core SDK has been configured in your project, and 3.3.2 Integrating HMS Core SDK of this service has been completed.

2.1 Create a motion capture engine.

// 自定义参数配置。 
Modeling3dMotionCaptureEngineSetting setting = new Modeling3dMotionCaptureEngineSetting.Factory() 
    // 设置检测模式。 
    // Modeling3dMotionCaptureEngineSetting.TYPE_3DSKELETON_QUATERNION:输出人体姿态对应的骨骼点四元数。 
    // Modeling3dMotionCaptureEngineSetting.TYPE_3DSKELETON:输出人体姿态对应的骨骼点坐标。 
.setAnalyzeType(Modeling3dMotionCaptureEngineSetting.TYPE_3DSKELETON_QUATERNION 
                        | Modeling3dMotionCaptureEngineSetting.TYPE_3DSKELETON) 
.create(); 
Modeling3dMotionCaptureEngine engine = Modeling3dMotionCaptureEngineFactory.getInstance().getMotionCaptureEngine(setting);

Modeling3dFrame encapsulates video frame or still picture data from the camera, and related data processing logic.

You can process the video stream by yourself, convert the video frame into a Modeling3dFrame object, which is used to detect the video frame, and supports the NV21 format.

Create a Modeling3dFrame object through android.graphics.Bitmap for the motion capture engine to detect images. The supported image formats include: JPG/JPEG/PNG.

// 通过bitmap创建Modeling3dFrame。 
Modeling3dFrame frame = Modeling3dFrame.fromBitmap(bitmap); 
// 通过视频帧创建Modeling3dFrame。 
Modeling3dFrame.Property property = new Modeling3dFrame.Property.Creator().setFormatType(ImageFormat.NV21) 
    // 设置视频帧宽度。 
    .setWidth(width) 
    // 设置视频帧高度。 
    .setHeight(height) 
    // 设置视频帧旋转角度。 
    .setQuadrant(quadant) 
    // 设置视频帧序号。 
    .setItemIdentity(framIndex) 
    .create(); 
Modeling3dFrame frame = Modeling3dFrame.fromByteBuffer(byteBuffer, property);

2.2 Call synchronous or asynchronous methods for motion capture detection.

asyncAnalyseFrame asynchronous method sample code:

Task<List<Modeling3dMotionCaptureSkeleton>> task = engine.asyncAnalyseFrame(frame); 
task.addOnSuccessListener(new OnSuccessListener<List<Modeling3dMotionCaptureSkeleton>>() { 
    @Override 
    public void onSuccess(List<Modeling3dMotionCaptureSkeleton> results) { 
        // 检测成功。 
    } 
}).addOnFailureListener(new OnFailureListener() { 
    @Override 
    public void onFailure(Exception e) { 
        // 检测失败。 
    } 
});

Sample code of analyseFrame synchronization method:

SparseArray<Modeling3dMotionCaptureSkeleton> sparseArray = engine.analyseFrame(frame); 
for (int i = 0; i < sparseArray.size(); i++) { 
    // 检测结果处理。 
};

2.3 After the detection is completed, stop the engine and release the detection resources.

try { 
    if (engine != null) { 
        engine.stop(); 
    } 
} catch (IOException e) { 
    // 异常处理。 
}

Demo

During the integrated development process, you can online bill of lading if you have any questions, and someone will answer it for you.

for more details >>

Visit Huawei Developer Alliance official website
Get development guidance document
Huawei Mobile Services open source warehouse address: GitHub , Gitee

us and know the latest technical information of HMS Core for the first time~