Yuque desktop technical architecture practice

Author: Yi Zhilin (Wei Jun)

As a productivity tool provided by Yuque for users, Yuque desktop has maintained high-frequency iteration and healthy business growth since its launch for more than two years. This time, we will mainly introduce some of our technical architecture thinking and practice when we are working on the desktop, and will also share some common desktop application solutions and experiences that we have accumulated.

The article will be divided into four parts. First, it will briefly introduce the Yuque desktop, then introduce the current application architecture and key points of the Yuque desktop, and then introduce several key architectural items in the architecture, and finally summarize.

Introduction to Yuque Desktop

Yuque is a note and document knowledge base tool hatched from the Ant Experience Technology Department. Two years ago, we launched the Yuque desktop client based on the characteristics of Yuque users and subsequent development strategies, aiming to provide creators with a better creative experience.

Compared with the products and services provided by existing browsers, the desktop products we provide mainly consider the following points:

• Interference-free : Give users an immersive authoring experience, unlike browsers that have other windows, tabs to interfere, and users' minds that run out of time.
• System-level resident : the opening speed is faster, you can start it with one key or use various shortcut tools to evoke it.
• Integrate more operating system capabilities : multiple windows, system menus and shortcut keys to improve creative efficiency, read and write files, and integrate with system software.
• Offline : It is expected to be able to create without obstacles in the case of offline or weak network.

Desktop Architecture Overview

The R&D test is mainly divided into three layers on the left. The bottom layer is the infrastructure of Yuque, which relies on a large number of cloud services provided or encapsulated by the backend of Yuque, as well as the underlying security capabilities and storage modules.

The middle layer is the encapsulation of some capabilities that are more partial to the application architecture. The above is the auxiliary capabilities used at the code level, as well as the modules of the main process, and then there are some management capabilities provided to the application and some functional modules related to UI. The top layer It is a business application built on the basis of the underlying architecture. Including several modules that are the core of desktop application comparison, and some business modules undertaken by sub-applications (the concept of sub-applications will be introduced in detail later)

At the same time, there are also a lot of dependencies on the right side to assist R&D to complete the release, quality and stability management of Yuque desktop.

Architecture Overview - Key Points

Compared with ordinary web applications, we think it is more important for the desktop to have the following capabilities: including security, software upgrades, and basic capabilities common to the desktop:

Architecture Overview - Security

Security is the lifeline of a productivity tool software, especially as a knowledge management tool, Yuque attaches great importance to security.

basic security

• When downloading the installation package, a security management mechanism is required to avoid malicious replacement during the download process;
• Upgrade to the latest Electron version (Yuque is currently following the official version, and will also refer to Microsoft's head application (vscode) to avoid scenarios that have not been considered;
• At the same time, the files downloaded to the local by the user offline, including pictures, attachments, etc., also need to be encrypted.

boot security

• Antivirus software : Startup security mainly refers to some security issues when starting the software, such as whether the binary module is signed, so as to avoid being checked and killed by antivirus software and cause it to fail to start. You can also contact the security vendor to add a whitelist, and at the same time, it can also improve the startup speed.
• Prohibit debugging : Because the software code will be downloaded to the client, you can prohibit the software from being debugged in the client browser.
• Database key management : The local database file needs to ensure that even if it is maliciously obtained, the content inside cannot be viewed. By generating a memory security level solution, we ensure that other non-current computer Yuque software cannot be opened even after getting the database file.

Application Security

• The rendering container sets a whitelist to control the introduction of malicious pages;
• The rendering process closes the Node function and turns on the isolation mode to avoid excessive rendering process permissions;
• Electron itself is a web development mode, so the security problems encountered in the web will also be encountered in Electron and can be handled uniformly.

Architecture Overview - Software Upgrade

Compared with the web, the client software has a very big difference. When there is a function update, there is an upgrade process. Unlike the web, you can directly refresh the page. As a product with rapid iteration, Yuque desktop has also stepped on a lot of pits for upgrading.

Yuque desktop consists of two parts: software packages including basic modules such as Electron and Node.js + and its own business code.

Mac OS : The upgrade process under Mac is relatively simple. After the software is downloaded, hdiutil is used to simulate the user's manual installation process, and the user can restart the installation to complete the installation.

Windows : Due to the particularity of the environment under Windows, after downloading the installation package, the installation interface is automatically opened through the main process, and the user is guided to install manually step by step.

In fact, this solution satisfied our early functional iterations very well, but as the number of users increased, we also encountered many problems.

for example:

• Huge bandwidth consumption for each upgrade: for each installation of each UV, there are nearly 100M downloads, and each time a version is pushed, an OSS traffic alarm will be encountered, which corresponds to the cost;
• Poor installation experience: Under Windows, because every upgrade is almost a new installation process, the experience is also relatively poor, and users often complain.

Therefore, we investigated an incremental update scheme. An Electron program includes Electron core package and business code. In fact, each change is only the business code, so theoretically, each update only needs to incrementally update the business code.

After downloading the incremental code directly to the incremental code under Mac, you can replace it.

It is more complicated under Windows, and we mainly encounter two problems:

1. File occupation problem : Due to the characteristics of the Windows system, if a file is in use, it cannot be deleted. So if you want to replace it, you must close the program, and then start it after deleting it. So we wrote an .exe executable to close programs, update files, and start Yuque.
2. UAB permission control : Another problem with file writing is that files in the C drive generally require authorization before they can be operated. There is no way for us to start the software to get such a high authority. However, a new PowerShell function has been added to Windows 7 and later. Through this function, it can guide users to authorize and get more advanced permissions.

Of course, there are still many detailed problems in the process, such as the problem of Chinese and English of the path in the replacement process, the problem of the user-defined environment variable location, etc.

Architecture Overview - Precipitation of Basic Capabilities

In addition, in the process of making software, we also precipitated some components that are not coupled with the business:

• Multi-window management: When providing users with more convenient multi-window editing capabilities, how to manage the opening, closing, performance monitoring, etc. of these windows;
• Webview: Different editors and sub-applications are carried by Webview, and a common module is required to maintain the life cycle of each webview used in the system;
• Offline and online: Computer offline and online status acquisition, although the browser provides this status acquisition and event monitoring, but it is not very accurate under Windows. We encapsulate a more general module.

Desktop architecture focus

From an architectural point of view, compared with the cool technology used, the more important thing is whether the R&D delivery efficiency is high, the performance is high, and the stability is high. We believe that the following three points are important criteria for judging the quality of the architecture.

Architecture Focus - Delivery Efficiency

In terms of desktop functions, including the editor, more than 60% of the functional modules are consistent with the web, so the isomorphic solution was used at the beginning.

Some experiences in the isomorphism process:

• The common code is moved to the Common directory: The Yuque code repository is in the monorepo mode. If there is no clear directory division, it is easy to cause cross-end compatibility problems. With this agreement, business R&D students will notice that this will be used on mobile or desktop
• Multi-terminal adaptation through webpack alias: This method is relatively common. For example, each terminal has different network request libraries, and adapter/request is used at the component level. When webpack is built, adapters map to different side implementations.
• Define multi-end code development specifications: sort out some differences between different ends, and avoid pits during development.

Delivery efficiency (isomorphism problem)

Although the isomorphic model can solve some of the problems we encountered at the time, with the increase of business scale and functions, some problems have been exposed one after another:

• Migrating to the Common directory, various dependency issues : Many functions have been launched before the desktop, and it takes a lot of time to migrate some complex components to Common.
• Lack of dynamic capabilities, lag in iteration (inconsistency between web and desktop functions) : After the component is released on the web, the desktop needs to be released to support it, so there are often complaints about the inconsistency between the web and the desktop.
• It is prone to multi-terminal compatibility problems (environmental dependencies, etc.) : Although we have defined some specifications, it is difficult to completely avoid environmental dependencies.
• The lack of an independent sandbox will easily affect the main application (memory leaks, styles) : the web may run out and be refreshed, etc. It is not easy to encounter problems, but because the desktop is resident, if there are some memory leaks or style pollution The problem directly affects the overall availability.

Delivery Efficiency (Sub-App)

Considering the above reasons, we upgraded the code reuse architecture to a sub-application mode, using the desktop container to nest an html online or offline page.

To put it simply, the sub-application mode can be understood as the various small program modules that Alipay enters into Jiugongge.

• Fast iteration : Provides independent release and iteration capabilities, so there is no need to follow the overall release of the desktop. Moreover, business students directly follow up the entire delivery process, without the need for desktop students to participate, improving delivery efficiency.
• With terminal-related capabilities : Each sub-application can use the JSBridge capabilities provided by the desktop side by default, and can naturally achieve the same capabilities as the desktop side modules.
• Independent sandbox : It is independent of the main window on the desktop, and uses the container on the desktop to complete the rendering, so it can completely isolate the process level, and the memory overhead between them is clear at a glance, which can better manage and control, and ensure the overall application stability.
• Loading initialization : In addition to some of the above advantages, there will also be some problems, such as slow loading speed, etc. We solve this problem to a certain extent through webview preheating, caching, etc.

Architecture Focus - Performance

Performance is a problem that desktop software must face, and it needs to be optimized continuously from different angles

It mainly includes these aspects:

• Startup speed optimization : The startup speed is the first image of the user. We mainly cache the main process code, display the loading as soon as possible to avoid white screen, and execute some tasks of the main window and the rendering process at the same time to achieve concurrent effects;
• Main process optimization : The main process and the rendering process are executed synchronously. If the main process does too many tasks, it will cause the user to freeze or even crash. So try to minimize what the main process does;
• Web optimization : At the same time, many optimizations that have been done on the web before can also be used. For example, lazy loading, merging modules, etc. (combining multiple modules has its own overhead);
• Webview optimization : For example, preheating the webview and taking some performance control measures to avoid loss of control.

performance (persistent tasks)

In fact, performance optimization does not mean that things can be completely solved, but a long-term process. Maybe when we add new features in the future, there is a memory leak problem in the code, which can easily lead to performance drop. So we also established some persistent mechanisms:

mainly include:

• Routine observation : In the development mode, establish the ability to observe performance indicators, and be aware of it;
• Automated tasks : There will also be automated tasks on a daily basis, simulating long-term use by real users, and discovering memory leaks and other issues in time;
• Performance market : For online performance levels, you can have a comprehensive perception ability, and you can focus on it during the grayscale release process.

Architecture Focus - Stability

Compared with the web, the stability of the desktop side is also more demanding.

From the point of view of the R&D process, we mainly have two things:

1. Unit testing, integration testing : use code testing to assist overall stability
2. UIA : Improve stability through UIA automated test regression that simulates user behavior, and finds abnormalities in time.

Note: UIA is an automation solution developed by Yuque engineers, see: Macaca MacOS

In addition, a stability dashboard and real-time alarms are established to perceive online performance.

In order to ensure the stability of each release and reduce the cost of regression, we use the agile R&D model of weekly preview release to decompose the integration risks brought by large release releases.

Summarize

• According to the current business volume and team experience, select the appropriate technical architecture;

  • Now there must be desktop architectures that are newer than Electron, such as flutter, tauri, etc., it is necessary to comprehensively look at such as team accumulation and stability, whether there are mature commercial products, etc.;
• Performance and stability optimization is an ongoing process, with metrics and perceptions established first;

• Delivery efficiency and delivery quality are most easily overlooked, but are important considerations for architectural solutions;

  • The criteria for judging whether the architecture is good or bad must be determined by the business effect. Delivery efficiency and delivery quality are one of the means to measure the business effect.