Create a third-party application development framework with the help of cloud development CloudBase platform

Lead

Tencent Lexiang is a one-stop corporate community launched by Tencent, providing online classrooms, exams, learning maps, live broadcasts, documents, questionnaires and other application services.

Tencent Lexiang started to dock the corporate WeChat open platform capabilities from 0. It has been online as a third-party application for corporate WeChat for 4 years. Nowadays, Tencent Lexiang has to become a service provider to allow other Saas products to be accessed as third-party applications. How can the accumulated experience be passed on to the developers who are about to access the application?

Tencent Lexiang’s third-party application open platform has been online for some time. The first phase of access includes Coral Points, Tencent Questionnaire, and Production Cloud. These three are already mature products and have their own open interfaces, so The problem is how to connect the open interfaces of the two platforms to get through. Among them, Tencent Cloud iPaas connects the Coral Points platform and the production cloud for us, and the Tencent Questionnaire is developed by the two platforms together to develop the transfer station service.

In this process, we found that the development colleagues inside and outside the team are repeating the ability to dock the Lexiang open platform, and the docking process will encounter various problems due to inconsistent understanding and implementation. In those years, we have stepped on the WeChat docking company The pit is spreading. At this time, various internal tweets of the company mentioned CloudBase cloud development. After understanding, it was found that this is suitable for building a third-party application development framework. If all application access uses a unified framework, both development efficiency and post-maintenance are There are great advantages.

Preliminary study of the problem

What needs to be done to access a third-party application?

accepts callback: company’s authorization/deauthorization notification for the application, company’s address book change notification, ticket push for calling credentials, etc., are all actively pushed to the application by the Lexiang service, and the application needs to provide cgi to receive the callback event. In particular, ticket is an important field in exchange for interface call credentials, so implementing receiving callback cgi is the first step to start development;
interface call: interface call seems to be a problem that SDK can solve, but the interface call of saas is not that simple. The interface has an application dimension and an enterprise dimension. The required credentials are different, and the credentials cannot be obtained frequently and need to be cached. The cache of enterprise credentials is also affected by enterprise authorization/deauthorization events. These logics are not satisfied by the SDK;
address book synchronization: pulls the interface and writes the DB. It seems that you can write business logic with your hands. In fact, it is also a good test of the basic skills of developers. Whether the synchronization process is smooth, whether the synchronization script can interrupt or resume running, how to clean up dirty data, etc. It is easy to miss these details without stepping on enough pits;
login authorization: standard OAuth protocol access, there are not many pits, but it still involves the development of a series of functions such as identity verification, authorization jump, authorization callback, token generation and issuance, repetitive and unchallenged work often makes People are tired.

The above points are only what needs to be done to connect to the open platform. From the perspective of web applications, in addition to selecting a server-side language for development, the overall architecture, database, cache, object storage and other basic components must also be considered. , Divide VPC, deploy applications, etc. Therefore, even with a more mature open platform, there are certain thresholds for many development teams. The development framework is to lower this threshold and attract more thoughtful development teams to implement their applications simply and quickly. In my initial conception, developers only need to implement the application’s own pages and interfaces, and don’t need to care about how the application interacts with the Lexiang backend. With the help of cloud development CloudBase's one-click deployment, they can become Lexiang’s third party. Application to provide services for thousands of enterprises. And this idea is now beginning to take shape.

framework implements

1. Database

CloudBase comes with a non-relational database, and its usage is similar to MongoDB, so we can configure the cloudbaserc.json to create the following data table when the application is initialized;

companies: company table, used to record the company information of the current application authorized from Lexiang, and it is also a necessary table for the saas application.
departments, users, department_user: three necessary tables in the address book, record the relationship between users and departments and the middle. The department_user intermediate table here is actually not necessary, because cloudbase provides a non-relational database, and the department information of each user can be recorded in the users table, but the final design still creates an intermediate table like MySQL, which is a farther one. The goal laid a small foreshadowing, which will be explained later.
lx_suite_callback_logs: records the logs of callbacks from Lexiang, which is convenient for locating problems;
lx_suites: mainly used to record the current valid ticket, the token used by the application to access the Lexiang interface, and the token used by each enterprise to access the Lexiang interface. It is actually equivalent to a cache, because CloudBase currently does not have the ability to directly mount redis, so First, borrow the database as a cache. Since the reading and writing of this table are all basic cloud functions that come with the framework, developers don't need to pay attention to it. Therefore, there are more elegant solutions that can be upgraded at any time.

In fact, the most important concept of this development framework, , is to transform the development model for the enjoyment interface into for database development. It is often said that web application development is based on the addition, deletion, modification, and checking of databases. This is the advantage brought by the framework.

2. Receive callback

The framework implements the cloud function base_suite_callback , which is responsible for monitoring ticket changes and notifications of enterprise opening/closing applications. The data will fall into the companies , lx_suites , and lx_suite_callback_logs tables. This cloud function is also the most important entry of the entire framework. Once it is unserviceable or the response timeout will have a serious impact, the instance will also be warmed up to avoid the loss of enterprise authorization information due to a cold start.

3. Interface call

The Lexiang platform has more than ten application modules, hundreds of application interfaces and more than ten address book interfaces. All interfaces are encapsulated under base_lx_apis , providing function descriptions, allowing developers to simply call Lexiang’s capabilities.

In addition to interface encapsulation, cloud functions also implement the access_token caching and refreshing mechanism, because the storage service needs to be used, which cannot be satisfied by a simple SDK.

Also need to pay special attention to is that the interface authorization mode provided by client credentials is 060be0a4dab78e, which is suitable for server-to-server requests. If customers directly request the interface, serious vulnerabilities will occur, and enterprise data will be maliciously tampered with, leaked, and other risks. . Using the cloud function permission settings provided by CloudBase, you can prohibit the client from directly calling the cloud function. The client must first call the developer's cloud function, which can be accessed through the server call, which ensures the security of the application. In the sample code of the framework, a code demo is also provided to guide developers to use it correctly.

4. Address book synchronization

The framework provides a fully synchronized cloud function base_sync_contact , which can be triggered by timing tasks or client calls.

The process of fully synchronizing the address book can easily cause data disorder due to code logic, script interruption and other reasons.

Here I introduced the concept of the address book version. For each synchronization, deprtments and department_user tables, but the data will not be modified. After the full synchronization is over, the data is consistent, and the current company address book will be updated. Version, delete the data of other versions in the table, to ensure that the address book data is available at every moment.

5. Login authorization

The identity verification logic is mainly placed in the client code Vue, and the company ID of the current user’s login status is judged in the form of routing middleware. If it is missing or does not match the company ID on the url, it will be redirected to the Lexiang authorization where the specified company is located. page. These code logics are valuable experience accumulated by Lexiang, and developers can directly use them to avoid various B-side login status errors.

6. Effect experience

Based on the encapsulation of the above basic capabilities, it is simple enough to develop applications on this framework.
We also try to develop a third-party learning material management platform, which only requires 3 steps:

(1) Create a new data table;

(2) Realize the cloud function logic related to material management, which will call the base_lx_apis cloud function to call the Lexiang interface, and also use the object storage API;

(3) Add related routing pages in Vue.

This is the same as developing any web application. Developers will not stop because they don’t know how to get through enjoyment. The cost of learning is transferred from learning to enjoy the interface, learning Saas application development to learning the use of CloudBase.

one-click deployment

One-click deployment is CloudBase Framework . Through simple operations, the code can be run in the cloud development environment. But for Lexiang to actively call back events to the cloud development environment, you also need to configure the receiving callback cgi corresponding to the cloud development environment on Lexiang; after configuring the cgi, you must first let Lexiang push the ticket to allow the company to authorize the application; login authorization is also It involves the configuration of the trusted callback domain name; the application entrance also needs to configure the link. Developers should look at the documentation, copy the link from the cloud development to the configuration on the Lexiang page, and then debug and verify each function to ensure that there is no manual copy error. Refer to the following figure for the process:

In order to solve the problem of manual initial configuration, for this scenario, an interface for applying initial configuration is provided on the Lexiang side. The cloud development environment can directly use suite_id+suite_secret to initialize the configuration of the specified fields, including callback cgi, login authorization trusted domain name, and entry address. After the configuration is completed, Lexiang will actively push the ticket to the application to ensure that the enterprise can directly authorize it. use. The call of this initialization interface is placed in the postDeploy hook of CloudBase one-click deployment, which does not require manual operation and ensures that the configuration is successful before the deployment is successful, which greatly reduces the barriers to use of the framework.

The current one-click deployment process is shown in the figure:

There are also several links in the entire one-click deployment process that can be optimized;

It is not necessary to click one-click deployment on github, and you can directly put the deployment button in the company management background of Lexiang, allowing public developers to jump directly from Lexiang to Tencent Cloud for deployment configuration.

Based on the optimization of the first point, if the cloud development deployment page allows configuration parameters to be passed through url or form, then the Lexiang side can directly bring it, eliminating the need for developers to copy and paste.

Steps 4 and 5 download the custom private key and inject it into the cloud function. At present, cloud development cannot be configured directly through cloudbaserc.json. It would be better if it can be supported later.

capability expansion

The original intention of the framework design is to allow developers to quickly build a third-party application from scratch. But with the thinking and understanding of the implementation process, I think the usage scenarios of this framework should not be limited to this, it should also serve the docking of other mature applications and platforms.

The docking between applications or platforms with open interfaces is often deadlocked due to interface inadequacy, and no one is willing to modify their interfaces to adapt to each other. The final solution can only be for one of the parties to make concessions to build an intermediate service from scratch, and look at the interface documents of both parties for development.

If Lexiang can provide this framework, what can the framework provide?

External authorization

The login authorization mentioned above means that Lexiang authorizes the service that deploys this framework (referred to as A service). But if the two platforms are docked, service A also needs to authorize the third-party platform. Although developers no longer need to implement the process of service A being authorized to log in, I think there is something else they can do to make secondary development easier, such as encapsulating the OAuth and smal protocols for external authorization.

callback notification delivery

In addition to the pre-implemented logic, Lexiang Callback retains configurable hooks, allowing developers to configure custom cloud functions or message queues. When developers use cloud functions or message queues to process them, they do not need to worry about whether the message is forged, expired, or replayed.

data write-through business database

The framework has realized that the address book data is written into the database of the cloud development environment from the Lexiang pull interface, but it is completely meaningless for business outside the cloud development environment (even if the VPC is opened, there is no SDK to read and write after leaving the cloud development environment. The database is too difficult to implement by yourself). Therefore, the framework should have the ability to write address book data into the specified database from the Lexiang pull interface, and should not only support non-relational databases such as MongoDB, but also support the most commonly used mysql, so the address book table design retains the middle department_user table. The framework should allow the mysql connection to be configured so that data can be written directly to the business database.

privatization deployment

As a Saas product, privatization deployment is an eternal topic. It is not easy for the product itself to be privatized. It is often necessary to consider the means of service downgrading to satisfy the privatization, not to mention the turn-on and turn-off function of third-party applications. But if it is a third-party application developed based on cloudbase, why not take it away with one-click deployment?

low code development

If a third-party application development framework is a starting point, then the next target may be a low-code development platform. No matter which method is used for development, the current encapsulation of each basic capability is necessary and reusable, and the first step is taken to bring the possibility for the next goal.

summary

From the development of enterprise WeChat third-party applications to the development of open platforms for third-party applications to access, identity replacement has given me a lot of experience. The emergence of cloudbase just made me think and summarize, and realize some design concepts. Welcome everyone to discuss and discuss with me.

PS: Starting from the design of this framework, I named this project "Enjoy Reemployment Program". Thinking about the day I retire, I will use this framework as a Lexiang service provider to do various outsourcing projects for Lexiang. Enjoy never retreat until you are eighty years old.

appendix

Github code repository:

https://github.com/TencentLexiang/cloudbase-template

product description

Cloud Development (Tencent CloudBase, TCB) is a cloud-native integrated development environment and tool platform provided by Tencent Cloud. It provides developers with highly available, automatically and elastically scalable back-end cloud services, including serverless capabilities such as computing, storage, and hosting. , Can be used for cloud integration to develop a variety of end applications (small programs, official accounts, web applications, Flutter clients, etc.) to help developers build and manage back-end services and cloud resources in a unified manner, avoiding cumbersome servers in the application development process With construction and operation and maintenance, developers can focus on the realization of business logic, with lower development thresholds and higher efficiency.
Open cloud development: https://console.cloud.tencent.com/tcb?tdl_anchor=techsite
Product documentation: https://cloud.tencent.com/product/tcb?from=12763
Technical document: https://cloudbase.net?from=10004
Technical exchange group, latest information, follow WeChat public account [Tencent Cloud Development CloudBase]