Tencent Two Sides: Now, if you want to implement an embedded monitoring SDK, how would you design it?

Hello everyone, I am Nian Nian!

This is a comprehensive design question that my friend was asked last week. If you haven't used the buried point monitoring system, or you don't have a deep understanding of it, it's basically cool.

The original text was first published on my official account: front-end private education every year

This article will make it clear:

1. What problems is the buried point monitoring system responsible for, and how should the api be designed?
2. Why use img's src to send requests, and what is sendBeacon?
3. How to deal with the error boundary of react and vue?

What is Buried Point Monitoring SDK

For example, a company has developed and launched a website, but it is impossible for developers to predict what will happen when users actually use it: How many pages have users browsed? What percentage of users will click the confirmation button of a pop-up window, and what percentage will click cancel? Are there any page crashes?

Therefore, we need an embedded monitoring SDK to collect data, and then perform statistical analysis later. With the analysis data, the website can be optimized in a targeted manner: pages with very few PVs should not waste a lot of manpower; pages with bugs should be repaired quickly, otherwise it will cost 325.

The more famous buried point monitoring is Google Analytics, in addition to the web side, there are SDKs for iOS and Android.

Reply to "ReactSDK" in the background of the official account to get the github of the react version

Scope of functions of buried point monitoring

Because of different business needs, most companies will develop a set of buried point monitoring systems, but basically they will cover these three types of functions:

User behavior monitoring

Responsible for counting PV (number of page visits), UV (number of page visitors), and user click operations.

This type of statistics is the most used, and with these data, we can quantify the results of our work.

Page performance monitoring

Although developers and testers will evaluate these data before going online, the user's environment is different from ours, maybe it is a 3G network, or it may be a very old model. We need to know the performance data in actual use scenarios, such as Page load time, white screen time, etc.

Error alarm monitoring

Only by obtaining wrong data and dealing with it in time can a large number of users be affected. In addition to the error information captured globally, there are also error warnings caught in the code, which need to be collected.

The following will start from the design of the api and further expand the above three types.

Design of SDK

Before starting the design, let's take a look at how to use the SDK

 import StatisticSDK from 'StatisticSDK';
// 全局初始化一次
window.insSDK = new StatisticSDK('uuid-12345');


<button onClick={()=>{
  window.insSDK.event('click','confirm');
  ...// 其他业务代码
}}>确认</button>

First mount the SDK instance to the global, and then call it in the business code. When creating a new instance here, you need to pass in an id, because this buried point monitoring system is often used by multiple businesses, and different data can be distinguished by id. source.

First implement the instantiation part:

 class StatisticSDK {
  constructor(productID){
    this.productID = productID;
  }
}

data sending

Data sending is one of the most basic APIs, and all subsequent functions are based on this. Usually, the front-end and back-end separation scenarios use AJAX to send data, but the src attribute of the image is used here. There are two reasons:

1. There is no cross-domain restriction. For example, srcipt tags and img tags can directly send cross-domain GET requests without special processing;
2. Good compatibility, some static pages may have scripts disabled, then the script tag cannot be used;

However, it should be noted that this picture is not for display. Our purpose is to "transmit data". We just use the src attribute of the img tag to splicing parameters behind its url, and the server receives it and then parses it.

 class StatisticSDK {
  constructor(productID){
    this.productID = productID;
  }
  send(baseURL,query={}){
    query.productID = this.productID;
    let queryStr = Object.entries(query).map(([key, value]) => `${key}=${value}`).join('&')
    let img = new Image();
    img.src = `${baseURL}?${queryStr}`
  }
}

The advantage of the img tag is that it does not need to be appended to the document, and the request can be successfully initiated by simply setting the src attribute.

Usually the requested url will be a 1X1px GIF image. The online articles are vague about why the returned image here is a GIF. Here are some materials and tests:

1. With the same size, the GIF size is the smallest among pictures in different formats, so choose to return a GIF, which has less performance loss;
2. If it returns 204, it will go to the onerror event of img and throw a global error; if it returns 200 and an empty object, there will be a CORB alarm;

Of course, if you don't care about this error, you can return an empty object. In fact, there are some tools that do this.

3. There are some buried points that need to be actually added to the page. For example, spammers will add such a hidden flag to verify whether the email is opened. If it returns 204 or 200 empty objects will cause an obvious image placeholder.

    <img src="http://www.example.com/logger?event_id=1234">

More elegant web beacon

This way of marking is called web beacon (web beacon). In addition to gif images, since 2014, browsers have gradually implemented specialized APIs to accomplish this more elegantly: Navigator.sendBeacon

easy to use

 Navigator.sendBeacon(url,data)

Compared with the src of the picture, this method has more advantages:

1. It will not preempt resources with the main business code, but will send it when the browser is idle;
2. And it can also ensure that the request is sent successfully when the page is unloaded, without blocking the page refresh and jump;

The current buried point monitoring tools usually use sendBeacon first, but due to browser compatibility, the src of the image is still needed.

User behavior monitoring

The api for data sending is implemented above, and now the api for user behavior monitoring can be implemented based on it.

 class StatisticSDK {
  constructor(productID){
    this.productID = productID;
  }
  // 数据发送
  send(baseURL,query={}){
    query.productID = this.productID;
      let queryStr = Object.entries(query).map(([key, value]) => `${key}=${value}`).join('&')
      let img = new Image();
      img.src = `${baseURL}?${queryStr}`
  }
  // 自定义事件
  event(key, val={}) {
    let eventURL = 'http://demo/'
    this.send(eventURL,{event:key,...val})
  }
  // pv曝光
  pv() {
    this.event('pv')
  }
}

User behavior includes custom events and pv exposure, and pv exposure can also be regarded as a special custom behavior event.

Page performance monitoring

The performance data of the page can be obtained through the performance.timing API. The obtained data is a timestamp in milliseconds.

You do not need to understand all of the above, but the more critical data are as follows, according to which the time points of key events such as FP/DCL/Load can be calculated:

1. Page first render time: FP(firstPaint)=domLoading-navigationStart
2. DOM loading complete: DCL(DOMContentEventLoad)=domContentLoadedEventEnd-navigationStart
3. Pictures, styles and other external link resources are loaded: L(Load)=loadEventEnd-navigationStart

The above values can correspond to the results in the performance panel.

Back to the SDK, we just need to implement an api that uploads all performance data:

 class StatisticSDK {
  constructor(productID){
    this.productID = productID;
    // 初始化自动调用性能上报
    this.initPerformance()
  }
  // 数据发送
  send(baseURL,query={}){
    query.productID = this.productID;
      let queryStr = Object.entries(query).map(([key, value]) => `${key}=${value}`).join('&')
      let img = new Image();
      img.src = `${baseURL}?${queryStr}`
  }
  // 性能上报
  initPerformance(){
    let performanceURL = 'http://performance/'
    this.send(performanceURL,performance.timing)
  }
}

Moreover, it is automatically called in the constructor, because the performance data must be uploaded, so the user does not need to manually call it every time.

Error alarm monitoring

Error alarm monitoring is divided into JS native error and React/Vue component error handling.

JS native error

In addition to the errors caught in try catch, we also need to report errors that are not caught - listen through the error event and the unhandledrejection event.

error

The error event is used to monitor DOM operation errors DOMException and JS error alarms. Specifically, JS errors are divided into the following 8 categories:

1. InternalError: Internal error, such as recursive stack explosion;
2. RangeError: Range error, such as new Array(-1);
3. EvalError: Error using eval();
4. ReferenceError: Reference error, such as using an undefined variable;
5. SyntaxError: Syntax error, such as var a = ;
6. TypeError: Type error, such as [1,2].split('.');
7. URIError: invalid parameter passed to encodeURI or decodeURl(), such as decodeURI('%2')
8. Error: The base class for the above 7 errors, usually thrown by developers

That is to say, the above 8 types of errors that occur when the code is running can be detected.

unhandledrejection

The error thrown inside the Promise cannot be caught by the error, in this case, the unhandledrejection event is required.

Back to the implementation of the SDK, the code to handle the error alarm is as follows:

 class StatisticSDK {
  constructor(productID){
    this.productID = productID;
    // 初始化错误监控
    this.initError()
  }
  // 数据发送
  send(baseURL,query={}){
    query.productID = this.productID;
      let queryStr = Object.entries(query).map(([key, value]) => `${key}=${value}`).join('&')
      let img = new Image();
      img.src = `${baseURL}?${queryStr}`
  }
  // 自定义错误上报
  error(err, etraInfo={}) {
    const errorURL = 'http://error/'
    const { message, stack } = err;
    this.send(errorURL, { message, stack, ...etraInfo})
  }
  // 初始化错误监控
  initError(){
    window.addEventListener('error', event=>{
      this.error(error);
    })
    window.addEventListener('unhandledrejection', event=>{
      this.error(new Error(event.reason), { type: 'unhandledrejection'})
    })
  }
}

Like initial performance monitoring, initial error monitoring must be done, so it needs to be called in the constructor. Subsequent developers only need to call the error method in the try catch of the business code.

React/Vue component error

Mature framework libraries will have error handling mechanisms, and React and Vue are no exception.

React's error boundaries

Error boundaries are the hope that when a rendering error occurs inside the application, the entire page does not crash. We set a bottom component for it in advance, and the granularity can be refined. Only the part where the error occurs is replaced by this "bottom component", so that the entire page cannot work normally.

Its use is very simple, it is a class component with a special life cycle, and it is used to wrap business components.

The two lifetimes are getDerivedStateFromError and componentDidCatch ,

code show as below:

 // 定义错误边界
class ErrorBoundary extends React.Component {
  state = { error: null }
  static getDerivedStateFromError(error) {
    return { error }
  }
  componentDidCatch(error, errorInfo) {
    // 调用我们实现的SDK实例
    insSDK.error(error, errorInfo)
  }
  render() {
    if (this.state.error) {
      return <h2>Something went wrong.</h2>
    }
    return this.props.children
  }
}
...
<ErrorBoundary>
  <BuggyCounter />
</ErrorBoundary>

An online sandbox was built to experience it, and the official account backstage responded with "error boundary demo" to get the address

Going back to the integration of the SDK, in the production environment, if an error is thrown inside a component wrapped by an error boundary, the global error event cannot be monitored, because the error boundary itself is equivalent to a try catch. Therefore, the reporting process needs to be done inside the component of the error boundary. That is, the componentDidCatch life cycle in the above code.

Vue's error boundaries

Vue also has a similar life cycle to do this, so I won't repeat it: errorCaptured

 Vue.component('ErrorBoundary', {
  data: () => ({ error: null }),
  errorCaptured (err, vm, info) {
    this.error = `${err.stack}\n\nfound in ${info} of component`
    // 调用我们的SDK，上报错误信息
    insSDK.error(err,info)
    return false
  },
  render (h) {
    if (this.error) {
      return h('pre', { style: { color: 'red' }}, this.error)
    }
    return this.$slots.default[0]
  }
})
...
<error-boundary>
  <buggy-counter />
</error-boundary>

Now we have implemented the skeleton of a complete SDK and dealt with how the react/vue project should be connected in actual development.

The SDK used in actual production will be more robust, but the idea is no different. If you are interested, you can read the source code.

Epilogue

The article is relatively long, but to answer this question, these knowledge reserves are necessary.

If we want to design the SDK, we must first understand its basic usage, and then we need to know how to build the code framework behind it; then we need to clarify the functional scope of the SDK: it needs to be able to handle three types of monitoring of user behavior, page performance, and error alarms; finally, react , Vue projects, usually do error boundary processing, how to access our own SDK.

If you think this article is useful to you, like and follow is my greatest encouragement!

Your support is the driving force for my creation!