Analysis of the highly scalable technical architecture of Deco editor

1. Background

The Deco manual intervention page editor is Deco workflow. The Deco editor realizes the visual arrangement of the output results of the Deco intelligent restoration link. The schema of the intelligent restoration output is modified in the Deco editor, and the final modified After the Schema is processed by the DSL, the object code is downloaded.

In order to empower the business and create a smart code ecosystem, the Deco editor not only meets the general static code download scenario, but also needs to be customized for different business parties. This must make the Deco editor architecture design more open and at the same time developing The level needs to be able to meet the secondary development scenario.

Based on the above background, the following goals are mainly pursued when designing the editor's architecture:

• The editor interface is configurable and can realize customized development;
• Realize the real-time update and rendering of third-party components;
• Data, state and view are decoupled, with high cohesion and low coupling between modules;

2. Business logic

2.1 Business logic analysis

The D2C Schema runs through the Deco workflow. The main job of the Deco editor is to parse the Schema, generate the layout and manipulate the Schema, and finally generate the code through the Schema.

Input: Schema json data after semantic processing

Reference: Schema json data after manual intervention

For the introduction of related Schema, please see Bump Technology Revealed·Deco Smart Code·Start the Revolution of Production and Research Efficiency .

2.2 Business structure analysis

Deco editor is mainly composed of navigation status bar, node tree, rendering canvas, style/attribute editing panel, panel control bar, etc.

The core process is the processing of the schema, so the core module is the node tree + rendering canvas + style/attribute editing panel.

The node tree and the style/attribute editing panel are relatively independent modules (the business logic is less doped, most of which are interactive logic), and can be developed as independent modules. The canvas part involves layout rendering logic and can be developed as a core module. Navigation status and panel control need to be handled as a core module.

After the business analysis is completed, we have an initial understanding of a business model of the editor, and it is very important to choose a suitable technical solution to implement such a business model.

3. Technical solution design reference

3.1 system.js + single-spa micro front-end framework

Based on the above analysis of the front-end business architecture, it is not difficult to think of the micro-front-end solution as soon as possible when designing the technical solution.

Split each business module in the editor into each micro-application, use single-spa to manage each micro-application in the integrated environment of the workbench. Has the following characteristics:

• Without refreshing, the same page can run applications with different frameworks;
• Front-end applications based on different frameworks can be deployed independently;
• Support lazy loading of in-app scripts;

shortcoming:

• It is difficult to manage the state between the application and the application, and you need to implement a state management mechanism by yourself;

The Deco editor does not currently have multiple application requirements. Fit index: ★★

3.2 Angular

Angular is a mature front-end framework with component module management and the following features:

• Built-in module management function, can pack different functional modules into a module;
• Built-in dependency injection function to inject functional modules into the application;

shortcoming:

• The learning curve is steep and unfriendly to new students who join the project;
• It is more complicated to load third-party components;

Fit index: ★★★

3.3 React + theia widget + inversify.js

Use inversify, a dependency injection framework, to inject different React Widgets , and each Widget can be independently packaged.

The writing method of Widget refers to the writing method of theia browser widget, which has the following characteristics:

• Widget represents a functional module, such as attribute editing module and style editing module;
• Widget has its own life cycle, such as corresponding hook processing methods when loading and unloading;
• Unified management of all Widgets through WidgetManager;
• Widgets are independent of each other and have strong scalability;

shortcoming:

• It is quite different from the traditional component construction method, which is quite challenging;
• APIs are many and complex, not easy to learn;

Fit index: ★★★★

3.4 React + inversify.js + mobx + global plug-in component loading

Use inversify to inject different plug-in components, each plug-in component is independently packaged, and mobx is used to manage the global state and state distribution.

The use of plug-in components has the following characteristics:

• Independent development of plug-in components, which can be asynchronously loaded to the global and rendered through configuration files;
• Plug-in components can share the global mobx state and update automatically through observer;
• Register plug-ins through Module Registry and manage plug-in loading in a unified manner;
• Naturally fits the loading and rendering methods of external business components;

shortcoming:

• The plug-in development model is more complicated and requires different services.

Fit index: ★★★★★

Based on the above technical solution design and reference, the global plug-in component solution was finally determined. The overall technology stack is as follows:

describe	name	characteristic
Front-end rendering	React	Currently supports dynamic loading of modules
Module management	inversify.js	Dependency injection, independent modules can inject various services
State management	mobx.js	Observables are automatically bound to component updates
Style processing	postcss/sass	Native css preprocessing
Package management	lerna	Easy to get monorepo
development tools	vite	Load module based on ES6 Module, extremely fast HMR

Ideas:

1. Build a general framework of core component modules and panel control, independent modules can be dynamically injected and rendered
2. Pull the module configuration file asynchronously, configure the rendering panel, and dynamically load the panel content
3. Independent modules are developed separately and managed by lerna
4. Business components (big promotion/quark) can be loaded as independent modules
5. Use dependency injection to manage various business modules to decouple data, state and view

4. Technical architecture design

Based on the above-identified technical solutions and ideas, the technical architecture of the editor is mainly divided into the following modules:

• ModuleRegistry
• HistoryManager
• DataCenter
• CoreStore
• UserStore

Use inversify.js for module dependency management, unified management through the Container mounted under the window:

Container is a container that manages various class instances. The class instance can be obtained in the Container through the Container.get() method.

Through the feature of inversify.js dependency injection, we inject HistoryManager and DataCenter into CoreStore, and use singleton mode when registering the module. The HistoryManager and DataCenter referenced in CoreStore or Container will point to the same instance, which is for the entire application. The consistency of the state provides a guarantee.

4.1 ModuleRegistry

ModuleRegistry is used to register each container in the editor, Nav, Panels, etc. Its main job is to manage containers (loading, unloading, switching panels, etc.).

The workbench is mainly divided into Nav container, Left container, Main container, Panels container:

Each container carries the corresponding front-end module. We designed a module configuration file module-manifest.json to load the corresponding js module file in each container:

{
  "version": "0.0.1",
  "name": "deco.workbench",
  "modules": {
    "nav": {
      "version": "0.0.1",
      "key": "deco.workbench.nav",
      "files": {
        "js": [
          "http://dev.jd.com:3000/nav/dist/nav.umd.js"
        ],
        "css": [
          "http://dev.jd.com:3000/nav/dist/style.css"
        ]
      },
    },
    "left": {
      "version": "0.0.1",
      "key": "deco.workbench.layoute-tree",
      "files": {
        "js": [
          "http://dev.jd.com:3000/layout-tree/dist/layout-tree.umd.js"
        ],
        "css": [
          "http://dev.jd.com:3000/layout-tree/dist/style.css"
        ]
      }
    }
  }
}

The ModuleRegistry processing flow is as follows:

4.2 CoreStore

CoreStore is used to manage the state of the entire application, including NodeTree, History, etc. Its main business logic is divided into the following points:

1. Get D2C Schema
2. Convert Schema to Node structure tree
3. Generate a new Node structure tree through operations such as modification, addition, deletion, and replacement
4. Push the latest Node structure tree into the History instance injected into CoreStore
5. Save the Node structure tree to generate a new D2C Schema
6. Get the latest D2C Schema download code

CoreStore injects instances of HistoryManager and DataCenter from the Container. The general usage is as follows:

import { injectable, inject } from 'inversify'
import { Context, ContextData } from './context'
import { HistoryManager } from './history'
import { Schema, TYPE } from '../types'

type HistoryData = {
  nodeTree: Schema,
  context: ContextData
}

@injectable() // 声明可注入模块
class Store {
  /**
   * 历史记录
   */
  private history: HistoryManager<HistoryData>
  /**
   * 上下文数据（数据中心）
   */
  private context: Context

  constructor (
    // 依赖注入
    @inject(TYPE.HISTORY_MANAGER) history: HistoryManager<HistoryData>,
    @inject(TYPE.DATA_CONTEXT) context: Context
  ) {
    this.history = history
    this.context = context
  }
}

In the above code block, the history record and the data center are injected into the CoreStore as independent modules. The modification of the corresponding instance here will affect the instance objects under the Container, because they all point to the same instance.

4.3 HistoryManager

HistoryManager is mainly used to manage user operation history information. Based on the dependency injection feature, it can be directly injected into CoreStore for use, and the latest instance Container.get()

HistoryManager is an abstract class with a doubly linked list structure. By saving data snapshots to each linked list node, it is convenient and quick to shuttle historical records. The difference from the ordinary doubly linked list is that when a node is inserted into the History linked list, the previous linked list node will relink a new branch.

4.4 DataCenter

The data center is an independent module used by the entire Deco editor to manage floor data. At first, it was only used to serve the application development of the editor itself. Later, in order to facilitate users to debug in the editor application, the data center officially adopted a functional method. Settled down.

The floor data is the real data used by the page node during data binding, and it is obtained through the data context of the current node. If these real data are bound to the original NodeTree, then our NodeTree will be a node tree that stores all the information, the logic is quite complex and redundant, and it is also an extremely difficult task when doing Schema synchronization. Therefore, we consider extracting the floor data separately into a module for management.

As shown in the figure below, ContextTree is the data node tree of the data context, which is bound to the nodes on the NodeTree one-to-one, and is bound together by location information (such as 0-0, representing the first child node of the root node), and The difference between NodeTree is that is a node tree with spatial relationship . For example, if the node at position 0-2 needs to be inserted into a context node, the context node at position 0-2 needs to be inserted into the child node at position 0 Go, and set the context node at position 0-2-0 as a child node of node 0-2. In the same way, if node 0-2 is deleted from ContextTree, node 0-2 needs to be deleted from node 0's child nodes, and node 0-2-0 is set as node 0's child node.

In this way, the data management module is removed from NodeTree, and DataCenter independently manages the data context of the page, which not only makes us more decoupled at the code level, but also precipitates the functional module of "data center". It is convenient for users to perform debugging work during data binding.

5 Technical difficulties

5.1 Module management

5.1.1 inversify

Through the above architecture analysis, it is not difficult to see that although the main business function logic of the Deco editor is relatively simple, the various modules are independent of each other and cooperate with each other to cooperate to complete the data, status, history and rendering update operations of the editor application. It is not enough to simply manage the modules of ES6 Module. As a result, we introduced inversify.js for module dependency injection management.

inversify is an IoC (Inversion of Control) library, which is a JavaScript implementation of AOP (Aspect Oriented Programming).

The editor uses the "Singleton" singleton mode, which is the same instance every time the class is retrieved from the container. It doesn't matter whether you obtain an instance from a dependency in a class or obtain an instance from a global Container. This feature provides a strong guarantee for the consistency of the application state of the entire editor. The natural advantage of AOP is module decoupling, which improves the scalability of editor applications to a certain extent.

For more information about AOP and IoC, please refer to the article , Practice of AOP and IoC in SNS Service 161bb1533512ec.

5.1.2 mobx

Thanks to the state update mechanism of mobx observer mode, state management and view update are more decoupled, which provides great convenience for editor state maintenance and module management. Different data states (such as AppStore and UserStore) are independent of each other and do not interfere with each other.

5.2 Search and update of page node tree

The page node tree (NodeTree) is an abstract tree designed for Schema. Its main function is to add, delete, modify, and check page nodes. At the same time, it also maps to the rendering module to update and render the page canvas, and finally through a conversion method. Converted to Schema.

NodeTree is the abstract performance of page nodes. When the page design draft is relatively large (such as the big promotion design draft), the node tree is also a very large abstract tree. When searching for nodes, if you pass a simple deep traversal Algorithms for searching will have a huge performance loss. In response to this situation, we perform index matching search by getting the location information of each node (such as 0-0), which basically achieves a harmless search. In addition, based on the React update mechanism, after NodeTree nodes are added or deleted, the index is automatically updated, eliminating the trouble of manually updating location information.

At the same time, based on the design of node location information, the spatial information maintenance of the data context node described above is realized.

5.3 Loading and rendering of third-party components

In Deco smart code 618 application , there is a reference to the process of Deco component identification. In Deco, a component sample (view) corresponds to a component configuration. Based on the diversity of component configurations, a component may have multiple samples. For the editor, the recommendation of similar components returned by the component recognition service actually returns the attribute configuration information of the component. As long as the editor finds the corresponding sample component configuration information, it can perform the corresponding replacement work. So, how are third-party components loaded?

At the beginning of the article, we introduced the plug-in development model. For the Deco editor, the third-party component is also a plug-in, so you only need to package the third-party component library into a UMD format JavaScript file, and in the module-manifest.json file Configure deps plug-in information, so that third-party components are loaded into the editor's global environment in the form of plug-ins.

At the same time, the editor stores a configuration table of third-party components. When the user replaces similar components, the configuration information of the corresponding sample is obtained through the configuration table and sent to the canvas module of the editor for rendering. By default, third-party components are developed using React, and the editor uses the React.createElement native method for component rendering when rendering.

// 组件配置信息数据结构
export interface AtomComponent {
  id: string
  componentName: string
  logicHoc: string
  type: string
  image: string
  name: string
  props: any
  pkg: string
  tableName: string
  value?: string | number
  children?: (Partial<AtomComponent> | string)[] | string
  propsComponent?: Partial<AtomComponent>[]
}

Currently, this configuration table is packaged in the code. In future versions of the editor, this configuration table will be integrated with the Deco open platform and open to users for editing. Loaded in the way of three-way configuration.

6 final

At present, Deco has supported the development of large-scale conference venues under the background of 618 and 11.11, and has opened up the internal low-code platform for one-click code construction and page preview. The dozens of floors built by Deco have been successfully launched online, and the efficiency has increased by 48%.

The Deco smart code project is an exploration of the “front-end intelligence” of OMicro Lab. We try to start from the entry point of design draft generation code (DesignToCode), and complement the existing design to research and development capabilities, and then Improve the efficiency of production and research. Many algorithm capabilities and AI capabilities are used to realize the analysis and recognition of design drafts. Interested children's shoes are welcome to follow our account "Bump Lab" ( knows , Nuggets ).

7 more articles

Design draft to generate code with one key, research and development intelligent exploration and practice

Helps Efficient Delivery of Personalized Venue on Double 11: Deco Smart Code Technology Revealed

Ultra-Basic Machine Learning-Principles