Debugging capability construction and practice based on LowCodeEngine

Author: Liu Juping (Xu Li)

Overview

business background

Low-code is becoming more and more popular due to the advantages of R&D efficiency and delivery. It also brings a lot of black box logic while reducing costs and improving efficiency. Existing low-code platforms generally lack user-oriented debugging capabilities. When users encounter problems in low-code construction, troubleshooting and solving problems strongly rely on the platform’s customer service to answer questions or the browser’s native debugging capabilities, resulting in non-front-end users using low-code. The cost of the platform is high. Therefore, we need to provide debugging capabilities that are more suitable for low-code platforms and reduce the cost of using low-code platforms.

Technical Difficulties

In the product technology system of Ali's low-code engine, on the one hand, we need to provide the basic debugging capabilities of the low-code platform, which are applicable to most low-code debugging capabilities, and need to consider the usage habits and learning costs of non-front-end users; on the other hand, due to the The target user types of low-code platforms are quite different, and the debugging capabilities required by different low-code platforms are also different. In addition to the basic debugging capabilities, we need to provide extended customization capabilities to help different low-code platforms quickly build suitable debugging capability.

technical details

In terms of technical architecture, through layered design, we divide the debugging system into different layers such as Client, Server, and Protocol to undertake different responsibilities, and do a unified convergence of protocols downward, and connect with the protocol and rendering system of the low-code engine. Up can also support a high degree of customization under different layers. In product design, we have completed the exploration of low-code debugging capabilities such as log viewing, error code location, review elements, data source viewing and modification, based on the standard protocol of the low-code engine. Specific practice and analysis: The debugging capability under the low-code engine system has been implemented in the middle and back-end platform A of the group for half a year. The total number of active users is 600, accounting for 20% of the total monthly activity of the platform, and the debugging during Q&A has been greatly reduced. percentage of related issues.

Introduction to Low-Code Engines

The low-code engine is a low-code R&D framework with powerful scalability for low-code platform developers. It is jointly produced by Alibaba Front-end Committee and Dingding Yida. Users can quickly customize a low-code platform that meets their business needs based on the low-code engine. At the same time, the low-code engine also provides easy-to-use customization and extension capabilities based on the standard low-code designer, which can meet the unique functional needs of the business.

Open source address: https://github.com/alibaba/lowcode-engine

Official website: http://lowcode-engine.cn/#/

Click "Read the original text" at the end of the article, you can jump directly to view

Low-code debugging background introduction

There are many low-code platforms in Alibaba, among which the low-code platform in the picture is used to develop middle and backend pages. Since this platform has been in existence for a long time, the number of users is relatively large, and the types of people it covers are also relatively wide.

The low-code platform has about 4,000 monthly active users, and these 4,000 people include front-end, back-end, test development, etc. Only about 30% of the users are front-end developers, and the remaining 70% are non-front-end developers. To help these 4,000 people use the low-code platform, the low-code platform provides two front-ends to answer questions full-time, helping users of the platform solve their problems.

We can see that Q&A students need to solve more than 400 Q&A work orders in a month, and these work orders contain a large number of usage problems and application debugging problems.

Then let's assume that if this low-code platform does not have front-end students to answer questions, is it possible that at least 400 users may not be able to use our low-code platform. I think it's very possible, in this case, if they encounter a problem and can't find a channel to solve the problem, so they have a bad impression of the low-code platform, they can easily abandon the low-code platform , it is even possible to ditch all low-code platforms.

So we can see that this low-code platform has two problems:

• The cost of answering questions is high . For non-front-end R&D students, they are very dependent on the support provided by answering students. This will lead to the more users of the low-code platform, the higher the cost of answering questions.
• High learning threshold : Even if there is sufficient support for answering questions, the use of low-code platforms for non-front-end R&D has higher learning costs, which makes it difficult for low-code platforms to be promoted on a larger scale.

Therefore, we need to reduce the cost of answering questions. In order to find a solution to reduce answering questions, let's first analyze the cases of answering questions.

This is a process in which students who answer questions communicate with users of low-code platforms in the process of answering questions.

At the very beginning, the user will tell the answerer that an error has been reported, and the answerer needs to help to see where the problem is. At this time, the user is generally required to provide steps that allow the answerer to reproduce the problem. For example, you need to provide the address of the error page. In most cases, these pages have permission control, and you need to find the corresponding permission administrator to add permissions. Even some projects are too troublesome to add permissions, so you can only view and locate the problem through screenshots or videos.

Therefore, the first problem exposed in this case is that permission control will make answering questions longer, and it will also make it more difficult to view and locate problems.

Let’s continue to look at the second part. After the permission is added, the user and the answerer often have to communicate the recurrence path. Since the communication process between the user and the answerer often has poor information, repeated communication is needed to clarify the steps of the reproduction. The answerer In order to reproduce the error scene, the problem can be found.

We can see that the second problem exposed here is that communication is time-consuming and cumbersome to reproduce.

After finding the problems, I found that these problems are likely to be some very simple problems for the front end, such as wrong use of punctuation marks, wrong use of Chinese and English symbols, etc. Because most users of low-code platforms are non-front-end developers, they have no front-end related foundation. Therefore, there is no way to solve problems that are very simple for the front-end, and you can only rely on the help of answerers. If there is a lack of answering questions and users without front-end foundation want to solve similar problems, the cost of learning is still very high.

Therefore, on the one hand, in order to reduce the communication cost of answering questions, on the other hand, in order to reduce the learning cost of non-front-end R&D students. We decided to provide low-code debugging tools for low-code platforms.

Building low-code debugging capabilities

First, let's take a look at the current state of debugging on this low-code platform. Here we have designed a page, which has no problem in the designer, but when we configure a wrong expression in a component, we will see an error on the page when previewing. For the front end, you can open the console to view the error message, the error message is as shown in the figure.

Can this error message help low-code developers find the problem? In fact, whether it is for front-end R&D personnel or non-front-end R&D, it is difficult to directly locate the problem.

On the one hand, this page is likely to have hundreds or thousands of nodes, and each node has many configurations. This error message does not tell us which component node the error code is configured in. Nor can the user open every node to check every configuration.

On the other hand, there are many places where the low-code platform can configure the code, which makes our code very scattered. We cannot know how and where the error code is configured through existing information. In other words, even the front end, it is difficult to find the error code.

In order to allow users of low-code platforms to solve such debugging problems by themselves, we need to provide a debugging tool that allows users to

• Intuitively know how to view the content of the error
• You can know where the error code is based on the content of the error
• After finding the error code, know how to modify the code to fix the problem.

With this tool, users of low-code platforms can solve problems by themselves, which greatly reduces the dependence on answerers.

For the first question, we need to let the users of the low-code platform know "how to view the error report". The solution we provide is that when an error is encountered, we use a conspicuous logo to tell the user that the page you developed An error has occurred, and how many errors are there. After the user sees the logo, they will click on the logo. When the user clicks, a low-code debugging tool will appear on the page to help the user view the information related to the error report.

In this debugging tool, we allow users to find information about errors more intuitively by displaying logs. And, slowly users know that we provide a tool to help them develop low-code pages. After encountering other problems, it will open the low-code debugging tool by itself, which is equivalent to exercising the user's development habits.

After the user knows how to view the error report, the next question is how to let the user know where the error code is through the ability of log output.

For this problem, we help users find error codes by providing corresponding error codes for different types of errors.

1. First of all, for different types of errors, we will provide a separate error code, and also provide the corresponding context information, such as what is the component ID, and what is the content of the expression configured by the user;
2. Of course, this information is not enough, we also need to show the context information to users in a more semantic way. Therefore, in order to better organize these error codes, the corresponding context information and the description text displayed to the user, we use the error code management background to manage these information.

With this information, we can display very detailed information in the log panel of the low-code debugging tool to help users find the error code.

For example: the error we saw in the console before, the content displayed in the log panel becomes: there is an error in the execution of the expression of a certain component, and the reason for the error is given, and the content of the wrong expression is given. , wrong component ID, etc. Users can search in the low-code platform through the component ID to find the corresponding component. You can also find the error code.

For non-front-end development, we help users find the wrong code is not completely solved the problem, we also need to tell them how to modify the code to solve the error.

In order to solve this problem, we have added an external link function on the basis of the error log. Users can click the external link to open a document corresponding to the error code. This document describes the problem solving steps in detail in the form of pictures and texts. . This allows users to find solutions to simple problems.

Of course, for some relatively complex error reporting cases, users may still be unable to solve the problem by themselves, and still need the help of the answering staff. In order to reduce the cost of back-and-forth communication between users and answerers, we also support users to upload page logs, which will carry key information of the page, such as log content, JSON source code generated by the build, environmental information, and so on. This reduces the processing time of tickets.

What is the problem that the second user will encounter? For most low-code products, the effect they see when designing the page and the effect they see when previewing are inconsistent in some cases. For example, when we design this page, we configure expressions for a component, and these expressions will not be calculated in the canvas. This can cause a problem. The effect seen in the design is inconsistent with the effect seen in the actual display.

For example, the button display here is different. So when we are previewing, if the effect is not what we expected, we want to know

• What is the configuration expression for this component?
• How is the copy displayed by this component calculated?

In order to solve these problems before, users need to go back to the designer, find the corresponding component, and check the configuration of this component to solve it. For some complex configurations, such as ternary expressions, users need to add Console in many places to view the value of the data source in order to find the problem. Therefore, in order to help users quickly locate the problem of a single component configuration, we provide a component review function.

Its effect is like this, when switching to the component review panel, the user can click on an element in the page, or click on the corresponding component in the outline tree, to see the detailed information of this component.

For example, here is a low-code development page, we need to check the display logic of a button in the page.

1. We can click this button to see the details of this component
2. The detailed information displayed first is the value calculated by this component according to the configuration. We can see that the source of the text displayed by this button is the value of the content parameter.
3. The second displayed information is the original content of the component configuration. Here we can see that the parameter value of content is actually configured with an expression, that is, the value of content is calculated according to the expression of state.text.
4. The third information we show is the context information that this component can access, such as what is the value of this.state, such as what functions are under this.utils, and so on.

In this way, users can view the configuration of component parameters step by step through this tool, and can also deduce its calculation process. If it does not meet expectations, they can quickly find and know how to solve the problem.

In addition to the two tools just mentioned, we also provide two debugging tools, the data source panel and the schema panel.

• Data source panel: The data source panel allows users to view the value of the page data source, and in order to facilitate the users of the low-code platform to debug the data source, the data source panel also supports modifying the value of the data source.
• Schema panel : For low-code platforms, all visual operations and configurations actually operate on a json object. We call this json object schema. We also provide a schema panel to view the schema used when the page is rendered source code.

The above is the low-code debugging capability we built for this low-code platform. But it can only solve the problem of one low-code platform. In order to help more low-code platforms solve similar problems, we also need to provide low-code debugging extension capabilities.

Low-code debugging extensions

In Ali, there are nearly 200 low-code platforms, and the low-code platform we just mentioned is just one of them. So do other low-code platforms need debugging capabilities? It is definitely needed, but what they need is not only the debugging capabilities we mentioned before, but they also need to customize the debugging capabilities suitable for their platforms according to the characteristics of their platforms.

for example,

• Some platform users are front-end users, and they expect to be able to debug through browser plug-ins, which is more in line with the development and debugging habits of front-end users;
• And some platforms provide the ability to build forms, such as Yida. More form-related debugging functions are needed;
• Some platforms just want to add a button to the debugging capability to help users find the entry for answering questions.

In order to meet the demands of different low-code platforms for debugging capabilities, we need to provide extended capabilities on the basis of low-code debugging capabilities, so that low-code platforms can customize the corresponding debugging capabilities.

• Plug-in: First, we need to plug-in the debugging capabilities. Plug-in means abstracting each debugging capability into a plug-in. In this way, it is convenient for different low-code platforms to choose different debugging capabilities, and the debugging capabilities can be plugged and unplugged, and the corresponding plug-ins can be referenced for the debugging capabilities required by the platform.
• Basic debugging capabilities : After the debugging capabilities are plug-in, we can use the previously built debugging capabilities as basic debugging capabilities, and deposit them into basic debugging capabilities, such as log debugging capabilities, element review debugging capabilities, and so on.
• Debugging extension capabilities : When the basic debugging capabilities do not meet the requirements of the low-code platform, the low-code platform needs to use the debugging and extension capabilities we provide to develop custom debugging plug-ins.
• Low-code platform developers : When we have basic debugging capabilities and debugging extension capabilities, developers of low-code platforms can develop the debugging capabilities of the platform based on these two capabilities.

First of all, developers of low-code platforms can check whether the existing basic debugging capabilities are suitable for their low-code platforms, and choose the basic debugging capabilities suitable for their low-code platforms.

For the part that the basic debugging ability cannot meet, the low-code platform needs to develop the corresponding debugging ability plug-in. After the custom debugging plug-in is developed, it can be combined with the basic debugging capabilities. In this way, low-code platforms have debugging capabilities that are more suitable for their own platforms.

for example:

• Low-code platform A chose the basic debugging ability of element review, and developed the form debugging ability based on the construction characteristics of its own platform.
• Low-code platform B chose the basic debugging capabilities of logs and developed chart debugging capabilities.

In this way, different low-code platforms can have debugging capabilities that are more suitable for their own platforms. This is the low-code debugging expansion capability we ultimately expect to achieve.

The next question is: how can we achieve low-code debugging scalability, and it can also be used on dozens or hundreds of low-code platforms.

Fortunately, before we built low-code debugging and expansion capabilities, most of Alibaba Group's low-code platforms were built based on low-code engines. That is to say, among the 200 low-code platforms in Alibaba, nearly 130 low-code platforms are based on low-code engines to realize low-code construction and rendering.

With this premise, we can stand on the shoulders of giants and serve more and more low-code platforms by providing the low-code engine system with standard low-code debugging capabilities and low-code debugging extension capabilities.

As we mentioned before, most low-code platforms, visual drag-and-drop and configuration are actually manipulating a json file. The low-code engine is no exception, and the low-code engine has also developed a standard protocol for this json file, which is the "low-code engine construction protocol". Therefore, the low-code platform built by the low-code engine will generate a json schema that conforms to the construction protocol after the visual operation.

This json schema browser cannot directly parse and render, so it also needs to rely on the "rendering engine" to parse the schema and draw low-code pages into the browser. So is there only one set of this rendering engine? Not really. Since the rendering engine also uses a large number of basic components or business components when drawing pages, the technology stacks used by these components may be different. In order to allow materials from different technology stacks to be rendered into the browser, rendering The engine also needs to implement multiple sets according to different technology stacks.

Therefore, we have multiple sets of rendering engines. For example, the React rendering engine, the Rax rendering engine, and the Vue rendering engine in the future.

With so much information about the rendering engine, what is the relationship between the rendering engine and debugging capabilities?

The debugging capability requires the rendering engine to obtain page information. Why do you say that, let's first look at the process of low-code page drawing.

1. First, the low-code platform generates a json schema describing the page through visual configuration.
2. When the page is rendered, the rendering engine will draw the page according to the content of the schema, and in addition to the json schema, the rendering engine also needs to know what the components and plugins described in the json schema are. Therefore, it is also necessary to provide the rendering engine with information such as components and third-party plug-ins that will be used by the entire page.
3. After the rendering engine has this information, the page will be drawn on the browser.

Does the browser know the content of the schema during the entire rendering process? And do you know the parameter values of each component? In fact, the browser doesn't know, because these execution and parsing processes are all done by the rendering engine. Then the debugging ability needs to know this information, which can only be obtained through the rendering engine.

• For example, you need to get the schema of the page through the rendering engine
• For example, you need to talk to the rendering engine to get the parameter value of a single component.

Therefore, the debugging capability needs to communicate with the rendering engine in some way, such as through an API. But as we mentioned before, the rendering engine implements multiple sets according to different technology stacks, and the API provided by each rendering engine may be inconsistent. Therefore, we cannot develop debugging capabilities based on the rendering engine's API.

Why can't it be said here? Imagine if there is such a low-code platform, it supports the construction of both PC pages and H5 pages, but the PC page may use the React version of the rendering engine, while the H5 page uses the Rax version of the rendering engine. Then the same debugging capability needs to be compatible with two different versions of the rendering engine API, or two sets may need to be developed. This is definitely unreasonable, and there needs to be a standard of communication between debugging capabilities and rendering engines.

The standard for this communication is the low-code debugging protocol. It smoothes out the API and implementation differences between different versions of the rendering engine by defining a standard for communication between rendering engines and low-code debugging capabilities. for example:

• It defines how the rendering engine actively informs debugging capabilities;
• It also defines how the debugging capability can actively obtain relevant information through the rendering engine.

The advantage of this is that

• For developers of low-code platforms, they can develop low-code debugging capabilities without paying attention to the rendering engine of the technology stack that the page uses. There is also no need to do any compatibility and adaptation work;
• For low-code engines, rendering engines of different technology stacks can support all developed debugging capabilities after implementing the low-code debugging protocol.

Next let's see how the debug protocol is defined.

First, before defining the protocol, let's clarify what the purpose of the protocol is. Its main purpose is to complete the development of debugging capabilities. The development of debugging capabilities is mainly divided into two aspects:

1.schema access and operation types

The first is the operation related to the acquisition or modification of the schema. For example, get the schema of the entire page, and get the parameters and status of a single component node. These operations are essentially getting or manipulating schemas.

The low-code platform based on the low-code engine construction, the generated schema is generated according to the low-code engine construction protocol specification. Therefore, our operation schema is to operate the structure described by the construction protocol. In order to make the debugging ability more convenient to operate the construction protocol, we divided the construction protocol into three layers.

• The first layer is to build the top-level structure of the protocol. In order to operate this layer of structure, we define the schema debugging domain in the debugging protocol. In this way, the json schema of the entire page can be obtained through the schema debugging domain or the json schema of the entire page can be modified.
• The second layer is to build the container structure in the protocol. The container is a special type of component, and some special capabilities are added on the basis of the component capabilities. For example, it has a separate custom method and data source. Therefore, we define the Container debugging domain to operate the container layer structure. In this way, we can modify the data source of the container or call custom methods in the container in the debugging capability.
• The third layer is to build a single component structure in the protocol. It is definitely necessary to operate a single component during debugging. For example, you need to know the calculated parameter values of the component, the original configuration of the component, and obtain the json schema of the component. So we define the Node debug domain to manipulate this layer of structure.

2. Auxiliary Type

In addition to getting or modifying schema related operations, we also need some auxiliary types of operations when debugging. For example, the debugging capability wants to open a new browser page, it needs to execute the corresponding expression in the browser window. We use the Runtime debugging domain to communicate with these expressions that need to be executed. For example, when we inspect an element, we need a mask on the page to identify which component the user is viewing. Therefore, we define the Overlay debug domain for related communication.

This way our low-code debugging protocol is defined. Next we look at an example of how we actually debug tool and render engine communication during debugging according to the low-code debugging protocol.

Auxiliary Type - Runtime Debug Domain

The first example is an auxiliary type field, the Runtime debug field. When the debugging capability expects to open a new page in a new browser window, it will send a JSON of the Runtime field. The expression field in this JSON defines the expression to be executed. When the rendering engine receives the JSON, it knows its intention through parsing, and executes the expression in the JSON. After the execution is completed, it also sends a JSON to notify the debug panel that the expression execution is complete.

schema operation type - Schema debug domain

The second example is about operating the low-code engine to build the domain of the top-level structure of the protocol, that is, the Schema domain. When the Schema changes, the rendering engine will send the JSON in the figure to the debugging tool, and the debugging panel will know the current page. The content of the page rendering changes, and the debugging capability needs to perform related operations, such as updating the content displayed in the schema debugging panel, updating the outline tree of the review panel, and so on.

The low-code debugging protocol has basically been introduced so far. Let's consider another question. We mentioned that this protocol is sent by the rendering engine, but is this suitable? In fact, it is not suitable, because for low-code platforms, the rendering engine used in the online environment and the development environment is the same, and it is problematic to have so many redundant codes in the online environment. In addition, this implementation scheme will also make the rendering engine have major changes, and will also make the page using the old version of the rendering engine unable to use the debugging capabilities.

So who sends the debug protocol, and how does it communicate with the rendering engine?

In order to solve this problem, we have added a role of an agent, so that the way the rendering engine notifies the debugging capability becomes, the rendering engine lets the agent know that the debugging protocol is to be sent through the event, and the agent sends the corresponding debugging protocol. When the debugging tool wants to obtain the content of the schema, the debugging protocol sent by the debugging tool is received by the agent. After receiving, the agent will call the API of the rendering engine to obtain the content of the schema. After getting the result, the agent sends the result to the low-code debugging tool through the debugging protocol.

The benefits of adding a proxy are:

• The rendering engine only needs to provide the relevant API, and does not need to make large-scale changes for debugging capabilities.
• Agents only exist when developing low-code platform pages so that they don't affect our online environment.

Let's look at an example of actual communication,

1. First of all, when the page is initialized, the agent will call the API of the rendering engine to register the log callback event, which is equivalent to telling the rendering engine, remember to notify me when there is a log;
2. When a log appears in the rendering engine, the callback function registered by the agent will be called, which is equivalent to notifying the agent that there is a log, and the log content is so-and-so.
3. After the agent receives the notification, it will send the corresponding debugging protocol, which is the Json object in the figure.
4. The low-code debugging ability will receive the corresponding protocol, and it can also perform related operations. Here, the new logs received are displayed.

We have defined the low-code debugging protocol and implemented the communication ability between the rendering engine and debugging capabilities.

Next, we also need to provide a container to display basic plugins and custom plugins.

Depending on the type of presentation, we divide the plugins into two types:

1. Tab type : such as the previous log panel. Corresponds to the yellow area in the figure.
2. Action type : When clicked, it will only perform some operations, such as uploading logs, such as jumping to the Q&A page, etc. These plugins are shown in the red area in the figure.

Users can choose to develop different types of plug-ins according to the debugging capabilities they need.

We have basically realized the basic debugging extension capabilities now. In order to make the low-code platform more convenient to develop the low-code platform debugging capabilities, we also provide a tool chain for debugging related research and development, so that users can initialize development plug-ins, debug plug-ins and Integrate plugins. In addition to the tool chain, we also provide a more convenient API to operate the debugging protocol, so that users do not need to write JSON when developing debugging capabilities.

In this way, our low-code debugging extension capability is completed.

Low-code debugging skills in practice

In Ali's mid-to-backend low-code platform, there are 300 monthly active users using debugging capabilities, and the platform's Q&A volume has been reduced by about 10%.

Some of these simple answers have basically disappeared. For example, due to cross-domain, some interface requests will have errors. Before, many students came to us to answer questions because of these kinds of questions, but now there are no such answers, and because of component expressions Configuration errors, resulting in page errors, this type of Q&A is also much less.

In addition, Ali's external commercial low-code platform, Dingding should be used. Since most of the construction scenarios are building form pages, they have also expanded the corresponding debugging capabilities according to the characteristics of the form pages. View form data and error requests can be supported.

At present, the low-code debugging capability is still in its infancy and needs continuous exploration and construction. Just like browsers provide rich debugging capabilities for front-end R&D, the low-code field will have very rich debugging capabilities in the future. These debugging capabilities can help more and more low-code developers solve the problems encountered in the process of low-code construction. question.

In the next step, we will build more basic debugging capabilities:

• For example, the JS code written by the user in the low code, we can view and debug the corresponding JS code in the debugging tool. rather than having to go back to the designer to view it.
• For example, you can view the data source request information of the page in the debugging tool.
• For example, you can view the execution of the page or container life cycle, etc.

In addition, we will provide more low-code debugging containers:

• For example, Web containers are suitable for use by non-front-end R&D students.
• For example, the browser plug-in container is suitable for front-end research and development, and is more suitable for the debugging habits of front-end research and development students.
• For example, the client-side debugging container is provided for some special debugging scenarios, such as mobile-side debugging scenarios.

Finally, we also need to enable more rendering engines to support low-code debugging capabilities, such as the rendering engines of different technology stacks in our low-code engine, the React version of the rendering engine and the Rax version of the rendering engine, and the future community-supported Vue rendering engine.