HMS Core machine learning service image super-scoring ability, based on deep learning to improve news reading experience

When reading information on mobile terminals, people have higher and higher requirements for high-resolution, high-quality images. However, limited by many factors such as network traffic, storage, and picture sources, users cannot easily obtain high-quality pictures. The problem of obtaining high-resolution pictures of mobile display devices needs to be solved urgently. Not long ago, the HMS Core News Demo App made a series of updates and optimizations for the reading experience of news verticals, including image over-score.

Image super-resolution (Super Resolution) refers to the process of recovering a high-resolution (HR) image from a given low-resolution (LR) image, and is an important research direction in the field of computer vision image enhancement. The HMS Core News Demo App uses the image super-resolution capability of the service machine learning service to solve the problem of unclear pictures when users watch news materials. The user clicks the picture on the news reading interface, and can see the picture menu displays "Use ML service for image super-scoring", and then click to quickly obtain high-quality pictures. It also supports downloading and saving of pictures. For images of the same size, the resolution is generally increased by 100%~300% after the image is over-segmented, which can effectively solve the pain point of the image that affects the user's browsing experience due to low resolution.

technical background

So, how is the image super-resolution capability of HMS Core machine learning service realized?

Generally speaking, for the problem of insufficient image resolution, the traditional solutions are mainly interpolation-based super-resolution reconstruction and degradation model-based super-resolution reconstruction.

Interpolation-based super-resolution reconstruction methods usually provide an overly smooth reconstructed image, and use nearest neighbor interpolation, bilinear interpolation, and bicubic interpolation to supplement the lost part of the pixel details, thereby enhancing the resolution of the image.

The super-resolution reconstruction method based on the degradation model starts from the degraded degradation model from high-resolution to low-resolution images, extracts key information in low-resolution images, and combines the prior knowledge of unknown super-resolution images. knowledge to constrain the generation of super-resolution images.

However, traditional solutions have problems such as high computational cost and unstable performance. With the widespread application of artificial intelligence, especially deep learning in computer vision, people have begun to explore the use of intelligent methods to overcome many drawbacks of traditional techniques, such as deep learning-based super-score algorithms .

The method based on deep learning is to use a large amount of training data to learn a certain correspondence between low-resolution images and high-resolution images. Then, the high-resolution image corresponding to the low-resolution image is predicted according to the learned mapping relationship, so as to realize the super-resolution reconstruction process of the image.

Technical advantages

The super-resolution algorithm of the HMS Core machine learning service is based on the deep neural network and relies on the hardware's neural network accelerator to provide 1x and 3x super-resolution capabilities suitable for mobile terminals. 1x super-resolution is to remove the compression noise under the condition of the same resolution to obtain sharper and cleaner pictures; 3x super-resolution can effectively suppress the compression noise, at the same time, it uses intelligent methods to amplify it, making it higher resolution, providing 3x magnification capability for clearer detailed textures.

In addition, the super-resolution algorithm relies on the powerful NPU chip of Huawei mobile phones, and it only takes less than 600 milliseconds to perform super-resolution of the largest 800x600 picture, which is nearly 50 times faster than pure CPU computing. The additional ROM and RAM consumption of the super-resolution API is also very small. It is built into Huawei mobile phones, which can effectively reduce the size of the application and make the application lighter.

It can be seen from this that the wide application of image super-resolution capability of HMS Core machine learning service in computer vision has the technical advantages of high image quality, high speed, and ultra-lightweight , which can effectively suppress compression noise and save storage and traffic. In the case of insufficient picture resolution, greatly improve the experience when viewing small pictures enlarged.

In addition to improving the reading experience in news reading scenarios, image super-resolution can also be applied to medical imaging, astronomical observation, biological information recognition and other fields. HMS Core's machine learning service image super-resolution capability will also continue to carry out technological innovations for more industries. Provide practical and efficient solutions.

Learn more details>>

Visit the official website of Huawei Developer Alliance
Get development guidance documents
Huawei Mobile Services Open Source Warehouse Address: GitHub , Gitee

Follow us to know the latest technical information of HMS Core for the first time~