From rookie to expert, HMS Core image segmentation service teaches you how to finely cut out images in complex backgrounds

Since 2021, the autonomous driving track has entered an outbreak period, and the industry has become a must for large manufacturers and start-ups. Many of these companies have adopted computer vision as the technical base for autonomous driving. Through image segmentation technology, the car can effectively understand the road scene and distinguish where the road is and where is the person. In addition to the field of autonomous driving, image segmentation technology often appears in other important scenarios, such as:

• Medical Image Segmentation: Helping Doctors with Diagnostic Tests
• Satellite imagery analysis: suitable for in-depth study of large amounts of image data
• Video entertainment apps: portrait cutout, avoid video barrage covering faces

Therefore, the application of image segmentation technology is very important and extensive. HMS Core Machine Learning Service Image Segmentation Service adopts an innovative semantic segmentation framework. This framework labels every pixel in the image, and even hairline details can be clearly and completely preserved. In addition, the image segmentation service also improves the processing capabilities of images of different quality and sizes. For the white edges that often appear in segmentation algorithms, a more structured learning training method is adopted to make the edges more natural.

So how can such a stable and refined segmentation capability be achieved?

development practice

1. Development preparation

For the configuration steps of Maven warehouse and SDK, please refer to the application development introduction in the developer website:
https://developer.huawei.com/consumer/cn/doc/development/hiai-Guides/image-segmentation-0000001050040109#section1658976113112?ha_source=hms1

1 Configure the Huawei Maven warehouse address.

buildscript {
repositories {
google()
jcenter()
maven {url 'https://developer.huawei.com/repo/'}
}
dependencies {
...
classpath 'com.huawei.agconnect:agcp:1.4.1.300'
}
}
 
allprojects {
repositories {
google()
jcenter()
maven {url 'https://developer.huawei.com/repo/'}
}
}

2 Add compile SDK dependencies

dependencies {
    // 引入基础SDK
implementation 'com.huawei.hms:ml-computer-vision-segmentation:2.1.0.301'
    // 引入人像分割模型包
implementation 'com.huawei.hms:ml-computer-vision-image-segmentation-body-model:2.1.0.303'
}

3 Add permissions in AndroidManifest.xml

// 使用存储权限
<uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

2. Development steps

@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
    setContentView(R.layout.activity_main);
    if (!allPermissionsGranted()) {
        getRuntimePermissions();
    }
}
private boolean allPermissionsGranted() {
    for (String permission : getRequiredPermissions()) {
        if (!isPermissionGranted(this, permission)) {
            return false;
        }
    }
    return true;
}
private void getRuntimePermissions() {
    List<String> allNeededPermissions = new ArrayList<>();
    for (String permission : getRequiredPermissions()) {
        if (!isPermissionGranted(this, permission)) {
            allNeededPermissions.add(permission);
        }
    }
    if (!allNeededPermissions.isEmpty()) {
        ActivityCompat.requestPermissions(
                this, allNeededPermissions.toArray(new String[0]), PERMISSION_REQUESTS);
    }
}
private static boolean isPermissionGranted(Context context, String permission) {
    if (ContextCompat.checkSelfPermission(context, permission) == PackageManager.PERMISSION_GRANTED) {
        return true; 
    }
    return false; 
}
private String[] getRequiredPermissions() { 
    try {
        PackageInfo info =
                this.getPackageManager()
                        .getPackageInfo(this.getPackageName(), PackageManager.GET_PERMISSIONS); 
        String[] ps = info.requestedPermissions;
        if (ps != null && ps.length > 0) { 
            return ps;
        } else { 
            return new String[0];
        } 
    } catch (RuntimeException e) {
        throw e; 
    } catch (Exception e) { 
        return new String[0];
    }
}

2 Creating an Image Segmentation Detector

MLImageSegmentationSetting setting = new MLImageSegmentationSetting.Factory()
// 设置为人像分割
        .setAnalyzerType(MLImageSegmentationSetting.BODY_SEG)
        .create();
this.analyzer = MLAnalyzerFactory.getInstance().getImageSegmentationAnalyzer(setting);

3 Create an "MLFrame" object through android.graphics.Bitmap for the analyzer to detect images

MLFrame mlFrame = new MLFrame.Creator().setBitmap(this.originBitmap).create();

4 Call the "asyncAnalyseFrame" method for image segmentation

// 创建一个task，处理图像分割检测器返回的结果。
Task<MLImageSegmentation> task = this.analyzer.asyncAnalyseFrame(mlFrame);
// 异步处理图像分割检测器返回的结果。
task.addOnSuccessListener(new OnSuccessListener<MLImageSegmentation>() {
    @Override
    public void onSuccess(MLImageSegmentation mlImageSegmentationResults) {.
        if (mlImageSegmentationResults != null) {
//获得从图片中分割出的人像前景图
            foreground = mlImageSegmentationResults.getForeground();
            preview.setImageBitmap(MainActivity.this.foreground);
        }
    }
}).addOnFailureListener(new OnFailureListener() {
    @Override
    public void onFailure(Exception e) {
        return;
    }
});

5 Change the picture background

// 从相册中获取图片
backgroundBitmap = Utils.loadFromPath(this, id, targetedSize.first, targetedSize.second);
BitmapDrawable drawable = new BitmapDrawable(backgroundBitmap);
preview.setBackground(drawable);
preview.setImageBitmap(this.foreground);

6 Change the picture background

MLFrame mlFrame = new MLFrame.Creator().setBitmap(this.originBitmap).create();

(demo demo video below)

more details>>

Visit Huawei Developer Alliance official website
Get development guidance document
Huawei Mobile Services open source warehouse address: GitHub , Gitee

us and know the latest technical information of HMS Core for the first time~