Rendering Engine Analysis - OpenHarmony JS UI Source Code Reading Notes

Author: Men Liu

Hongmeng is a new generation of terminal operating system developed by Huawei, which can be applied to various types of devices such as IoT, watches, mobile phones, Pads, and TVs. The OpenHarmony 2.0 Canary version will be released in early June 2021, open source code for more subsystems, and support devices with more than 128MB of memory. This includes the new version of the JS UI framework. This article focuses on analyzing this part of the code. (The content of the article is for reference only. If there is any inaccurate description, thank you for your discussion and correction in the background~)

Hongmeng system overview

System Architecture Layering

It is recommended to go to the OpenHarmony official website [1] for more information. The following is the official technical architecture diagram:

To sum up, it is divided into four parts: application layer, framework layer, system service layer, and kernel layer. The kernel layer is mainly Linux of the macro-kernel and LiteOS of the micro-kernel, as well as various hardware drivers. The framework layer is closely related to the system service layer. There are horizontal frameworks and vertical services, which are subdivided into many relatively independent subsystems. The UI framework and the Ability framework are here.

Note that this is the architecture of OpenHarmony. It does not contain GMS and HMS, does not contain AOSP and does not support Android API. It is different from the version of HarmonyOS pushed by Huawei mobile phones. The comparison is made below.

Differences between HarmonyOS and OpenHarmony

The Chinese names are all called "Hongmeng", but there are still some differences behind them. HarmonyOS is a commercial operating system developed by Huawei, and OpenHarmony is an open source version donated by Huawei to the Open Atom Foundation (OpenAtom). The "sleeves" are wobbly.

I tried to organize the relationship between the concepts, as shown in the figure below. unofficial picture, for reference only, everything is subject to the official caliber .

First of all, Hongmeng apps are packaged and distributed in the HAP (Harmony Ability Package) format, which is a compressed package like Android APK, and the package structure is similar. After it is distributed to the terminal, it is unified by the API of Hongmeng (concept), and then it is divided into different modes. The version of Hongmeng pushed on Huawei mobile phones can be compatible with Android applications without feeling, and it must be dependent on AOSP. The open source part is mainly for low-end IoT devices, or watch bracelets, etc., even without a screen. The open source version in June 2021 just supports devices with a memory of more than 128MB, and cannot independently support applications on mobile phones. These are the two extreme scenarios of "shelling" and "self-research", which are not necessarily stable final states. Between the two, the subsystems of the framework layer and the system service layer in the overall architecture are relatively independent and decoupled, and can be used in these scenarios at the same time, but the functions are not sound enough, and some of them have been open sourced, such as Microkernel and JS UI framework, and some are not open source. Compatibility with Android is not a long-term solution, and the open source part has simple functions, so will the uncertain part in the middle be the main link in the future? Will Hongmeng really develop into a self-developed full-platform operating system in the future? Let's wait and see.

Then you can try to answer the most discussed Hongmeng questions in the community:

self-developed by Hongmeng? There must be a self-developed part, and it must also rely on some (foreign) third-party codes. require all codes to be self-developed at this stage. It cannot be completed in one or two years. It must first stand on the shoulders of giants to innovate, and then further reduce dependencies and deal with agreements and copyright issues.

Does Hongmeng rely on AOSP? It can be said to be dependent, or it can be said to be not dependent. HarmonyOS compatibility mode is dependent on the AOSP , or it will not be compatible Android application; open source OpenHarmony is not dependent on the AOSP , but mainly used in some low-end IoT devices.

Unless otherwise specified, " " mentioned in the following refers to OpenHarmony 161f11d26b58c0.

Harmony's UI Subsystem

There is no Java UI in Hongmeng's open source version (at least I didn't find it), the framework for implementing JS UI is called ACE, which is short for Ability Cross-platform Environment. There are two warehouses: ace_engine_lite [2] corresponds to the old ACE architecture, ace_ace_engine [3] Corresponding to the new ACE architecture, the implementation principle and upper-level syntax of the two are different.

Source directory structure

The OpenHarmony open source code is hosted on gitee, and there are more than 200 warehouses under the group. It is recommended to download the code according to the documents obtained from the official source code. For the corresponding relationship between the source code directory structure and each warehouse, please refer to the configuration file of the manifest warehouse [4].

If your environment is already configured, execute the following two lines of commands:

repo init -u https://gitee.com/openharmony/manifest.git -b master --no-repo-verify
repo sync -c

The whole process takes about 15~30min. After executing the first line of command, you can find the manifest configuration file and remove the dependencies of linux and some third_party, which will be faster.

The abbreviated version of the directory structure is as follows (skip irrelevant directories and expand only relevant parts):

OpenHarmony/
  ├── applications          # 应用程序样例
  ├── base                  # 基础软件服务子系统集
  ├── docs
  ├── domains               # 增强软件服务子系统集
  ├── drivers               # 驱动子系统
  ├── foundation            # 系统基础能力子系统集
  │   ├── aafwk                # Ability 框架
  │   ├── ace                  # JS UI 框架
  │   │   ├── ace_engine           # ACE 1.0, 仓库名: ace_ace_engine
  │   │   └── ace_engine_lite      # ACE 2.0, 仓库名: ace_engine_lite
  │   ├── appexecfwk           # 用户程序框架和包管理模块
  │   └── graphic              # 图形库相关的子系统
  ├── interface
  │   └── sdk-js               # JS API 的 TS 类型文件
  ├── kernel                # LiteOS 内核
  │   ├── liteos_a
  │   └── liteos_m
  ├── test
  └── third_party

New and old ACE framework

The following is the architecture diagram of the ACE framework [5]. Now the UI is described in the document and supported in the IDE. It provides a DSL[6] similar to the applet to develop the UI, HML (HarmonyOS Markup Language) + CSS + JS.

It looks similar to the architecture of the mainstream JS + native rendering framework. It provides a compilation tool to compile the source code into a js bundle. At runtime, there is a framework.min.js file that is executed by default after the JS engine starts to implement the node construction logic, and then A set of UIKit is implemented in the C++ layer, and there are about twenty or thirty components supported.

In another warehouse without lite suffix (ace_ace_engine), there is also a set of UI system implementation. The architecture level is more concise and the technology is more advanced. It should be a new generation of UI framework. This is the architecture diagram of the new ACE framework [7] ]:

The main body of this architecture is divided into application, framework, engine and cross-platform adaptation. The application layer is the syntax exposed to developers. There are several modes, which are explained in detail below. The framework layer implements the common componentization and MVVM capabilities of front-end frameworks, and can update the UI responsively. The following is the JS engine, which uses QuickJS and should also support V8. Then down is the rendering engine, which contains the core rendering pipeline, animation, events and various layout drawing algorithms.

The bottom porting layer is the key to adapting to multiple platforms. It defines the platform-independent layer data structure, which can be submitted to different compositors for composite rendering. From the code point of view, it supports Flutter Engine.

Overall rendering process

Container creation and management

The basic unit of Hongmeng UI is FA (Feature Ability), which corresponds to an instance of AceAbility and provides life cycle hooks. When OnStart() is executed, an AceContainer will be created internally. But this AceContainer is not directly held by AceAbility, but managed by the globally unique singleton AceEngine.

AceContainer provides various capabilities for UI rendering. It is a general management class that provides life cycle and function scheduling interfaces. It is divided into many sub-modules:

• Frontend : The execution environment of the front-end code, JS or JSON, with this layer of abstraction, it may also support other scripting engines.
• TaskExecutor : Task manager within a single thread, similar to TaskRunner in other rendering engines.

<!---->

• AssetManager : Resource manager, which can be used to load resources such as JS code, pictures, fonts, etc. Should also have caching capabilities.
• PipelineContext : The management class of the rendering pipeline, listening to the vsync callback to refresh the layout, drawing and rendering of the internal dirty nodes.

<!---->

• AceView : The UI root node generated by rendering can be pasted into the outer container.
• PlatformResRegister : The registration and management of platform resources, as well as some communication interfaces, can realize the same layer rendering function here.

JS UI development framework

In the Frontend section, there are three different types of implementations, namely imperative UI based on rendering instructions, declarative UI, and UI rendered with JSON files without a scripting engine. Both imperative UI and declarative UI use QuickJS as the script engine. The declarative UI contains the abstraction of V8, but this part does not seem to be open sourced.

JS Frontend (imperative UI)

"Important UI" is from the perspective of engine implementation, because the bottom layer is based on rendering instructions, and the syntax that is really revealed to developers can be connected to declarative and responsive front-end frameworks, such as Vue.js. The JS UI of Hongmeng official website corresponds to JSFrontend. The syntax is very similar to that of the applet. It should be inherited from the previous quick application. Each component corresponds to the template file of xx.hml, the style file of xx.css, and the style file of xx. js script file and xx.json configuration file.

<div class="container">
  <text class="title-text">{{headTitle}}</text>
</div>

/* xxx.css */
.container {
  flex-direction: column;
  margin-top: 20px;
  margin-left: 30px;
}
.title-text {
  color: #1a1a1a;
  font-size: 50px;
}

// xxx.js
export default {
  data: {
    headTitle: 'Capture the Beauty in This Moment'
  }
}

The above code will be packaged and compiled into a js file, loaded and executed dynamically.

When initializing the JS execution environment, a series of native interfaces are bound first, then the built-in js framework is executed, and then the user's js code is dynamically loaded and executed. In the qjs_engine.cpp [8] file of js frontend, you can see the interface registered with the JS environment:

Because QuickJS supports ES6 Module, these interfaces are registered in the module of ace, which can be imported by import, and the environment also has the global environment of ace:

import * as ace from 'ace'

// 创建 body 节点
ace.domCreateBody(0, 'div', {/* attrs */}, {/* styles */}, {/* events */})

The code that actually sends rendering instructions in ACE is in TaskCenter.ts[9] of third_party_jsframework. The design of the interface and the format of rendering instructions are basically the same as the TaskCenter.js of alibaba/weex. The simplified DOM API is also simulated with JS. Connected to the mini-program style DSL, there are already a lot of practices in this kind of technical community, so I won't introduce it anymore.

Declarative Frontend (Declarative UI)

There is not much public information in this regard. From the code point of view, it is not to use HML + CSS to build UI, but to combine UI with atomic layout functions like Flutter. In Flutter, Widget and Hongmeng's declarative JS UI are used. JSView is used. Combined with the code in jsproxyClass.js for analysis, it provides the decorator/annotation-assisted programming in the ECMAScript specification. The estimated code is written like this:

@Component
class MyDemo {
  Column(
    Text('Hello World!'),
    Text('Some text here')
      .fontsize(36)
      .color(Color.Red)
  ).center()
}

Here is the source code of JSView, tiled as shown below, there are not many types at present.

The advantage of this method is that the composition is better, the layout nodes will not affect each other like CSS, and the layout algorithm is more efficient. However, the layout method of HTML + CSS that the front-end loves is abandoned, and the existing front-end frameworks, component libraries, and style libraries cannot be directly migrated, and ecological construction will be somewhat difficult.

Card Frontend (scriptless UI)

It's a bit strange to call it CardFrontend. It may be mainly used in card-based scenarios such as Hongmeng Desktop Widget. It does not depend on the scripting engine, but is driven by a JSON file in a specific format.

There is no clear document on the format of the JSON file. Here is a test case. You can see that the main body is divided into template, styles, actions, and data. The data binding of curly braces can be written in the template {{some.data}} , you can write simple JS expressions in it, and it is not completely static. And there is UpdateData() interface on CardFrontend class, which can update the data of template, and has certain dynamic ability.

Node build process

The above three front-end frameworks correspond to different UI development methods, but the node construction and layout drawing algorithms in the second half are the same link, and they are self-drawn and rendered in C++, which does not depend on the system UI, and the consistency is relatively good. . The complete node construction process is drawn as follows:

The JSView of the declarative UI is directly bound to the JS environment and dispatched directly by the page/card code. JSView has a large number of derived classes, which correspond to the atomic style descriptions. The JSView tree will generate a Component tree, and the nodes are basically in one-to-one correspondence. For example, JSGrid will generate GridLayoutComponent, and JSText will generate TextComponent.

Both JSFrontend and CardFrontent synthesize rendering instructions (C++), one is generated by the front-end framework diff, the other is directly parsed from JSON, and then builds an internal simplified DOM tree (C++). Note that this so-called DOM tree is not bound to In the JS environment, it cannot be directly operated through the JS API. Each DOM node generates another Component node.

Component's derived class RenderComponent can directly generate RenderNode, which is a shorter link. For example, ImageComponent inherits from RenderComponent. Another derived class is ComposedComponent, which is composed of other Component. Element is also a similar data structure, derived from RenderElement and ComposedElement. Component can build Element and RenderNode, and Element can build RenderNode. Frankly speaking, I haven't understood the difference between these two trees. Can Component and Element be merged in the future?

RenderNode is the real layout node. Other rendering engines are called RenderObject or LayoutObject, which derives many subclasses and implements various layout algorithms. The type design is similar to Flutter's Widget. RenderNode lays out and then draws, and the output data structure is Layer Tree. This layer is used for cross-platform in Hongmeng, which is the Porting Layer[7] in the official architecture diagram. It can be crossed by submitting to different synthesizers. different platforms.

What is more interesting is that there is flutter in Hongmeng's third_party, and a FlutterSceneBuilder [10] is implemented. The data structure of the porting layer is basically the same as the layer of the Flutter Engine, and the code of ohos_layers is also added to the flow. In other words, Hongmeng's new ACE framework may run on Flutter Engine, and Flutter Framework may also run on Hongmeng's Graphic UI.

layout drawing rendering

The core class that implements the rendering pipeline is the PipelineContext, which stores multiple queues of dirty nodes. When the node attributes change, the layout and drawing will not be triggered immediately, but will be added to the cached queue. At the same time, it monitors the system vsync signal, flushes the dirty nodes recorded inside in sequence in each frame, and executes animation detection, node reconstruction, re-layout, re-draw, cursor refresh and other processes in turn.

The core code is the two paragraphs in the screenshot, which are simplified. Each frame of vsync will trigger the OnVsyncEvent() callback. If the surface has been initialized, the internal dirty nodes will be refreshed in turn. If it contains animation, it will call RequestFrame() to request the next trigger. One frame vsync. First, dirty Elements will trigger Rebuild() to rebuild the RenderNode, and call MarkNeedLayout() to add it to the queue to be laid out, then FlushLayout will refresh the queue, call the node's OnLayout() method to calculate the layout, and then add itself to the In the queue to be drawn, then FlushRender() calls the Repaint() method of the node in turn to redraw to generate a layer tree.

Discuss the development of Hongmeng

A picture was posted at the Hongmeng Developer Conference. The number of mobile phones, tablets, and PCs has not grown significantly for a long time, and there will not be much increase in the next five years. However, there are more and more IoT devices, such as watches, offline large screens, and car machines, and the number of IoT devices is still in a stage of rapid development. Hongmeng believes that the future is the era of the Internet of Everything, and the goal is to create a "distributed operating system for multi-smart terminals and all scenarios". Compared with AliOS, Hongmeng was born at the right time and has enough space.

The micro-kernel system, hardware drivers and subsystems of Hongmeng system are pluggable, which can be well compatible with IoT devices ranging from low-end to high-end with very different shapes. Another feature is the cross-device transfer of Ability. Taking advantage of Huawei's advantages in communication technology, multiple hardware devices can be directly connected at the system layer, which may lead to some new gameplay.

For new things, we always use old things as an analogy, and what we see is only the same parts as old things, and what we do not see is the real value. If Hongmeng is only compatible with Android and has no differentiated functions, it will not have much development significance, and only solve the feeling of "localization". It is also difficult to say that Android/iOS has any revolutionary subversion in technology compared to the previous generation of Nokia Motorola. Hongmeng's design concept still has some bright spots. system-level capabilities requires a process to be perceived by the application layer. may be just the beginning. In order to bring about a wave of explosive development, a person like Jobs may be needed to transform the new system capabilities into a new generation of interaction and a new generation of products. Considering the trend of national policies and technology self-research, the future of Hongmeng can be expected.

* Reference link

• OpenHarmony official website: https://www.openharmony.cn/
• ACE Lite: https://gitee.com/openharmony/ace_engine_lite
• ACE Engine: https://gitee.com/openharmony/ace_ace_engine
• Configuration file: https://gitee.com/openharmony/manifest/blob/master/default.xml
• ACE Architecture Diagram: https://gitee.com/openharmony/ace_engine_lite/blob/OpenHarmony-2.0-Canary/figures/JS%E5%BA%94%E7%94%A8%E5%BC%80%E5%8F %91%E6%A1%86%E6%9E%B6.png
• Applet-like DSL: https://device.harmonyos.com/en/docs/apiref/js-framework-file-0000000000616658
• New ACE architecture diagram: https://gitee.com/openharmony/ace_ace_engine/blob/OpenHarmony-2.0-Canary/figures/JS-UI%E6%A1%86%E6%9E%B6%E6%9E%B6% E6%9E%84.png
• qjs_engine.cpp: https://gitee.com/openharmony/ace_ace_engine/blob/OpenHarmony-2.0-Canary/frameworks/bridge/js_frontend/engine/quickjs/qjs_engine.cpp#L1925-1940
• TaskCenter.ts: https://gitee.com/openharmony/third_party_jsframework/blob/OpenHarmony-2.0-Canary/runtime/main/manage/event/TaskCenter.ts#L127-133
• FlutterSceneBuilder: https://gitee.com/openharmony/ace_ace_engine/blob/OpenHarmony-2.0-Canary/frameworks/core/pipeline/layers/flutter_scene_builder.h

follow us, 3 mobile dry goods & practice every week for you to think about!