1

Content source: HUAWEI Developer Conference 2021 HMS Core 6 AI Technology Forum, the keynote speech "End-Cloud Collaboration, HUAWEI HiAI Foundation Builds an Easier AI Computing Platform".

Speaker: Huawei HiSilicon AI Technology Expert

Hello everyone! HHUAWEI HiAI Foundation, as the basic platform for chip enablement, is based on a three-tier open architecture of "core, device, and cloud" to build a fully open smart ecology, helping developers quickly utilize AI processing capabilities and provide users with a better smart application experience. Taking full advantage of Huawei's Kirin chip capabilities, HUAWEI HiAI Foundation avoids network delay and congestion in real-time; in terms of privacy, it protects local pictures, videos, voice information, and operating habits; in terms of cost, developers can maximize the use of chips Computing power, saving computing costs and bandwidth costs. Since its release in 2018, the number of calls to third-party applications of the HUAWEI HiAI Foundation has increased year by year, and the current number of calls per day has exceeded tens of billions, which has been recognized by developers.

HUAWEI HiAI Foundation open framework

The reason why the HUAWEI HiAI Foundation is popular is that it supports more mainstream reasoning frameworks. While leading the construction of end-side AI industry standards, through open source, more developers can optimize models flexibly and efficiently.

In the first half of 2021, we have completed the docking with KwaiNN, Bytenn, MNN, TNN and other frameworks. On this basis, HUAWEI HiAI Foundation can support a lot of services and applications in Huawei mobile phones and various product lines. Such as video editing, live video recording, online payment, maps, input methods, etc.

Through the above framework diagram, we can see that HUAWEI HiAI Foundation, as the basic platform, is mainly composed of two parts: an inference acceleration platform and a heterogeneous computing platform. The function of the reasoning acceleration platform is to complete the docking with more upper-level reasoning frameworks to help developers focus on model optimization; the heterogeneous computing platform better enables computing IP such as NPU, CPU, GPU, DSP, and makes full use of the computing power of the chip. After that, HUAWEI HiAI Foundation can realize the unified multi-platform empowerment of Huawei products based on Kirin chips such as mobile phones, tablets, smart screens, speakers, etc., and deep integration with HarmonyOS, playing an active role in the end-side AI capability in the Hongmeng ecosystem.

Advantages of HUAWEI HiAI Foundation's End-Cloud Collaboration Solution

After HUAWEI HiAI Foundation went live, we still encountered many problems. For example, after a new application feature is released, the performance of the business operator needs to be continuously optimized and pushed along with the ROM system version updates such as mobile phones, making the overall online cycle longer. Some models will not be able to enjoy the latest performance optimization after the support service (EOS) is discontinued. As a result, the developers' demands are also very clear, that is, to follow the rhythm of the Internet to help the business quickly release and go online and cover more models, including EOS models.

In order to solve the above-mentioned typical problems and several other problems raised by the developers during the docking, the HUAWEI HiAI Foundation has undergone major upgrades in two aspects this year. One end of collaborative cloud computing-based independent library upgrade capability . Supports dynamic upgrade of the computing library, ensuring that the optimized operator library is pushed to the end side faster, and at the same time, the model is compiled through the plug-in to maintain a better heterogeneous strategy and high-performance fusion template. At present, this capability has been launched in the HarmonyOS 2.0 version to meet the rapid business launch needs of partners.

second is the ease of use upgrade . We will provide richer interface calls, user manuals, Demo demonstrations and toolkits that are easier to understand. Specifically, it mainly includes the following four points:

  • Model Zoo helps developers quickly enable NPU acceleration . Model Zoo integrates the hardware optimization model library, model optimization guidance, and the ability of the technical exchange community. Through it, developers can quickly know which types of models run faster on hardware, and how to optimize the model structure to obtain better energy efficiency benefits, etc. . In addition to making it easier for developers to participate in discussions and exchanges, the technical exchange community also provides a foundation for subsequent university scientific research cooperation projects and AI technology competitions.

  • with high performance, low power consumption, high precision and small model. Including model quantification toolkit and model structure search toolkit. The former has introduced 16bit, 8bit, 4bit, 2bit and adaptive hybrid quantization. The model is small and flexible, which can ensure that the model calculation progress is not lost. The latter is more through automatic search methods to help developers adjust the model structure in combination with hardware information to make it more agile and fast.

  • , GPU, NPU and other IPs 161cae02c27fdd. The following figure lists a typical data flow. From video decoders to image preprocessing, to image processing, rendering optimization, display, etc., a lot of IP calculations have been carried out. So how do these IPs collaborate efficiently in longer data streams? This requires automatic heterogeneity of multiple IPs. It can automatically integrate and optimize computing nodes, share computing memory, and achieve zero copy of data between multiple IPs. In addition, multi-IP scheduling tasks can be automatically scheduled to support priority control and automatic memory allocation between nodes.

  • model visualization-user model information is visible, improving the tuning efficiency . In the model structure visualization tool (Netron), developers can directly see the model structure of HUAWEI HiAI Foundation, which is more convenient.

Today, HUAWEI HiAI Foundation has joined the AI standardization organization AITISA (Artificial Intelligence Industry Technology Innovation Strategic Alliance), and participated in the drafting of end-side AI standards to jointly build AI industry standards. In order to allow developers to make better use of end-side AI capabilities, HUAWEI HiAI Foundation will maintain its original intention and build a better and easier-to-use end-side AI computing platform, serving and empowering developers, and improving application experience, thank you!

for more details>>

Visit Huawei Developer Alliance official website
Obtain the development guide document
Huawei mobile service open source warehouse address: GitHub , Gitee

and learn about the latest technical information of HMS Core for the first time~


HarmonyOS_SDK
596 声望11.7k 粉丝

HarmonyOS SDK通过将HarmonyOS系统级能力对外开放,支撑开发者高效打造更纯净、更智能、更精致、更易用的鸿蒙原生应用,和开发者共同成长。