Build front-end monitoring to collect N gestures of user behavior

Hello everyone, my name is Yang Chenggong.

In the previous article, we introduced in detail how the front end collects abnormal data. The purpose of collecting abnormal data is to monitor the operation of online projects at any time, and find problems and repair them in time. In many scenarios, in addition to anomaly monitoring, it also makes sense to collect user behavior data.

How to define behavioral data? As the name implies, it is the behavioral trajectory generated by the user in the process of using the product. For example, which pages you have been to, which buttons you have clicked, how long you have stayed on a certain page, and how many times you have clicked a certain button, you can record it if you need it.

However, recording behavior data is a matter closely related to business. It is impossible to record every step of every user's operation in great detail, which will generate extremely huge data, which is obviously unrealistic.

A reasonable approach is to evaluate according to the actual situation of the product, which modules and buttons need to be recorded in detail, and can be collected in more detail; which modules do not need to be focused, simply record the basic information.

According to this logic, we can divide behavioral data into two categories:

1. General data
2. specific data

The following sections describe how to collect these two types of data.

General data

In a product, the most basic behavior of users is to switch pages. Which functions the user uses can also be reflected from the switching page. Therefore, the general data is generally generated when the page is switched, indicating that a certain user has visited a certain page.

Page switching corresponds to the front-end route switching. You can get the data of the new page by monitoring the route change. Vue monitors routing changes in the global routing guard, and any routing switch can execute the callback function here.

 // Vue3 路由写法
const router = createRouter({ ... })
router.beforeEach(to => {
  // to 代表新页面的路由对象
  recordBehaviors(to)
})

React implements the same functionality in the component's useEffect. However, it should be noted that monitoring all route changes requires all routes to pass through this component for monitoring to be effective. The specific method is to add * configuration when configuring the route:

 import HomePage from '@/pages/Home'
<Route path="*" component={HomePage} />,

Then listen for route changes in the component's useEffect:

 // HomePage.jsx
const { pathname } = useLocation();
useEffect(() => {
  // 路由切换这个函数触发
  recordBehaviors(pathname);
}, [pathname]);

In the above code, the recordBehaviors() method is called and the parameters are passed in when the route is switched. Vue passes a routing object, React passes the routing address, and then you can collect data in this function.

Now that we know where to collect data, we also need to know what data to collect. The most basic fields for collecting behavioral data are as follows:

• app : the name/id of the app
• env : Application environment, generally development, testing, production
• version : the version number of the application
• user_id : current user ID
• user_name : Current username
• page_route : page routing
• page_title : page name
• start_at : entry time
• end_at : departure time

In the above fields, the application ID, environment, and version number are collectively referred to as application fields, which are used to identify the source of the data. Other fields are mainly divided into three categories: user , page , and time . Through these three types of data, one thing can be easily determined: who has visited which page, and how long they stayed there.

Configuration and acquisition methods of application fields We built front-end monitoring in the previous section. How to collect abnormal data? As mentioned in the above, I will not do any redundant introduction. The methods of obtaining fields are all common.

The following describes how to obtain other types of data.

Get user information

Modern front-end applications store user information in basically the same way, one in localStorage and one in state management. Therefore, the user information can be obtained from either of these two places. Here is a brief introduction to how to get it from state management.

The easiest way is to directly import the user state in the js file where the function recordBehaviors() is located:

 // 从状态管理里中导出用户数据
import { UserStore } from '@/stores';
let { user_id, user_name } = UserStore;

Here @/stores points to the file in my project src/stores/index.ts , which represents the entry file of state management, and replaces it with the actual location of your own project when using it. In the actual situation, there will also be the problem that the user data is empty. It needs to be dealt with separately here, so that we can see the difference in the subsequent data viewing:

 import { UserStore } from '@/stores';

// 收集行为函数
const recordBehaviors = ()=> {
  let report_date = {
    ...
  }
  if(UserStore) {
    let { user_id, user_name} = UserStore
    report_date.user_id = user_id || 0
    report_date.user_name = user_name || '未命名'
  } else {
    report_date.user_id = user_id || -1
    report_date.user_name = user_name || '未获取'
  }
}

In the above code, it is firstly judged whether there is user data in the state management, if so, get it, if not, specify the default value. The details of specifying the default value here should be noted, not casually specified. For example, the default value of user_id has the following meanings:

• user_id 为 0 : Indicates that there is user data, but there is no user_id field or the field is empty
• user_id 为 -1 : indicates that there is no user data, so the user_id field cannot be obtained

User data is often an error-prone place as it involves complex issues such as login status and permissions. After the above default values are specified, it is possible to determine whether the user status of a certain page is normal from the collected behavior data.

Get page information

Earlier, we called the recordBehaviors function and passed in the parameters where we listened for routing changes. The page information can be obtained from the parameters. Let's first see how to obtain it in Vue:

 // 路由配置
{
  path: '/test',
  meta: {
    title: '测试页面'
  },
  component: () => import('@/views/test/Index.vue')
}

// 获取配置
const recordBehaviors = (to)=> {
  let page_route = to.path
  let page_title = to.meta.title
}

Vue is relatively simple, and you can get page data directly from parameters. In contrast, React's parameter is just a routing address, and it needs to be processed separately to get the page name.

Generally, when designing permissions, we will maintain a set of routing data on the server, including routing addresses and names. The routing data is obtained after logging in and stored in the state management, so with the pathname, the corresponding routing name can be found from the routing data.

 // React 中
import { RouteStore } from '@/stores';

const recordBehaviors = (pathname) => {
  let { routers } = RouteStore; // 取出路由数据
  let route = routers.find((row) => (row.path = pathname));
  if (route) {
    let page_route = route.path;
    let page_title = route.title;
  }
};

In this way, the page_route and page_title fields of the page information are also obtained.

set time

In the behavior data, two fields start_at and end_at represent the time when the user enters and leaves the page, respectively. These two fields are very important, and we can judge a lot of information when we use the data later, such as:

• How long does a user stay on a page?
• Which pages does a user stay on during a certain period of time?
• During a certain period of time, which page has users staying the longest?
• Which users have the highest usage rate of a certain page?

There is a lot of information that can be judged based on these two time fields. The start time is easy to handle, and the current time is directly obtained when the function is triggered:

 var start_at = new Date();

There are many things to consider when it comes to the end time. First of all, when will the data be reported? Do users report when they enter the page, or when they leave the page?

If it is reported when entering the page, it can be guaranteed that the behavior data will be recorded and will not be lost, but the end_at field must be empty at this time. In this case, you need to adjust the interface when you leave the page, and update the end_time of this record. The implementation of this method is more troublesome:

 // 进入页面时调用
const recordBehaviors = () => {
  let report_date = {...} // 此时 end_at 为空
  http.post('/behaviors/insert', report_date).then(res=> {
    let id = res.id // 数据 id
    localStorage.setItem('CURRENT_BEHAVIOR_ID', id)
  })
}

// 离开页面时调用：
const updateBehaviors = ()=> {
  let id = localStorage.getItem('CURRENT_BEHAVIOR_ID')
  let end_at = new Date()
  http.post('/behaviors/update/'+id, end_at) // 根据 id 更新结束时间
  localStorage.removeItem('CURRENT_BEHAVIOR_ID')
}

In the above code, when entering the page, first report the data, save the id, leave the page and then update the end time of this data according to the id.

If you report when leaving the page, make sure that the reporting interface has been triggered before leaving the page, otherwise data loss will occur. If this precondition is met, the reporting logic will become as follows:

 // 进入页面时调用
const recordBehaviors = () => {
  let report_date = {...} // 此时 end_at 为空
  localStorage.setItem('CURRENT_BEHAVIOR', JSON.stringify(report_date));
}

// 离开页面时调用
const reportBehaviors = () => {
  let end_at = new Date()
  let report_str = localStorage.getItem('CURRENT_BEHAVIOR')
  if(report_str) {
    let report_date = JSON.parse(report_str)
    report_date.end_at = end_at
    http.post('/behaviors/insert', report_date)
  } else {
    console.log('无行为数据')
  }
}

Comparing these two schemes, the disadvantage of the first one is that the interface needs to be adjusted twice, which will double the amount of interface requests. The second scheme is called only once, but requires special attention to reliability processing. In general, the second scheme is better.

specific data

In addition to general data, in most cases we also collect some specific behaviors on specific pages. For example, whether a key button has been clicked and how many times it has been clicked; or whether the user has seen a certain key area, and how many times it has been seen (exposed), etc.

There is also a more professional name for collecting data - burying points. The intuitive understanding is that where data needs to be reported, a reporting function is buried in it.

Generic data is automatically collected for all pages, and specific data needs to be manually added according to the actual needs of each page. Take a button as an example:

 <button onClick={onClick}>点击</button>;
const onClick = (e) => {
  // console.log(e);
  repoerEvents(e);
};

In the above code, we want to record the click of this button, so we made a simple buried point --- call the repoerEvents() method in the button click event, which will collect data and report it internally.

This is the most primitive way of burying points, directly putting the reporting method into the event function. The repoerEvents() method receives an event object parameter, and obtains the event data to be reported in the parameter.

Many fields of specific data are the same as general data, and the basic fields required to collect specific data are as follows:

• app : the name/id of the app
• env : Application environment, generally development, testing, production
• version : the version number of the application
• user_id : current user ID
• user_name : current username
• page_route : page routing
• page_title : page name
• created_at : trigger time
• event_type : event type
• action_tag : Behavior ID
• action_label : Behavior description

Among these basic fields, the first 7 fields are exactly the same as the previous general data acquisition, and will not be repeated here. In fact, there are only 3 dedicated fields that need to be obtained for specific data:

• event_type : event type
• action_tag : Behavior ID
• action_label : Behavior description

These three fields are also very easy to obtain. event_type indicates the type of event trigger, such as click, scroll, drag, etc., which can be obtained in the event object. action_tag and action_label are attributes that must be specified, which represent the identification and text description of this buried point, which are used to facilitate access and statistics in subsequent data processing.

Now that we know what's going on with collecting specific data, let's implement it in code.

Manual burial point reporting

Assuming that we want to embed the login button, according to the above data collection method, we write the code as follows:

 <button data-tag="user_login" data-label="用户登录" onClick={onClick}>
  登录
</button>;
const onClick = (e) => {
  // console.log(e);
  repoerEvents(e);
};

In the code, we pass two identifiers, tag and label, through the custom attribute of the element, which are used to obtain it in the reporting function.

The code logic of the reporting function repoerEvents() is as follows:

 // 埋点上报函数
const repoerEvents = (e)=> {
  let report_date = {...}
  let { tag, label } = e.target.dataset
  if(!tag || !label) {
    return new Error('上报元素属性缺失')
  }
  report_date.event_type = e.type
  report_date.action_tag = tag
  report_date.action_label = label

  // 上报数据
  http.post('/events/insert', report_date)
}

In this way, a basic specific data buried point reporting function is implemented.

Global automatic reporting

Now let's go back and sort out the reporting process. Although the basic functions have been realized, there are still some unreasonable things, such as:

• An event handler must be specified for the element
• A custom attribute must be added to the element
• Manually adding buried points in the original event handler is highly invasive

First of all, our method of burying points is based on events, that is to say, whether the element itself needs event processing or not, we have to add it to it, and call the repoerEvents() method inside the function. If a project needs to bury a lot of places, the access cost of this method will be very high.

Referring to the logic of abnormal monitoring before, let's change the idea: Can global monitoring events be automatically reported?

Think about it, if you want to monitor events globally, you can only monitor the events of elements that need to be buried. So how do you determine which elements need to be buried?

Above we have specified two custom attributes for the buried element: data-tag and data-label . Is it OK to judge based on these two custom attributes? Let's try it out:

 window.addEventListener('click', (event) => {
  let { tag, label, trigger } = event.target.dataset;
  if (tag && label && trigger == 'click') {
    // 说明该元素需要埋点
    repoerEvents(event);
  }
});

The above code also judges a custom attribute dataset.trigger, which indicates that the element needs to be reported when the event is triggered. This flag is required for global listener events to avoid event conflicts.

After adding a global listener, collecting specific data for an element is simple, as follows:

 <button data-tag="form_save" data-label="表单保存" data-trigger="click">
  保存
</button>

Experiments have proved that the above global processing method is feasible. In this case, there is no need to add or modify the event handler function on each element, but only need to add three custom attributes to the element data-tag , data-label , data-trigger can automatically report data buried points.

component report

The above global monitoring event reporting method is much more efficient than manually burying points. Now let's change the scene.

Under normal circumstances, when the buried point function is mature, it will be packaged into an SDK for other projects to use. Wouldn't it be a good way to let developers listen to events globally if we implemented the collected data according to the SDK's idea?

Obviously not very friendly. If it is an SDK, then the best way is to aggregate all the content into a component, and implement all the reported functions in the component, instead of letting users add listener events to the project.

If you encapsulate a component, then the function of the component is best to wrap the element to be embedded, so that the custom element does not need to be specified, but is converted to the component's property, and then the event monitoring is implemented in the component.

Taking React as an example, let's see how to encapsulate the above collection function as a component:

 import { useEffect, useRef } from 'react';

const CusReport = (props) => {
  const dom = useRef(null);
  const handelEvent = () => {
    console.log(props); // {tag:xx, label:xx, trigger:xx}
    repoerEvents(props);
  };
  useEffect(() => {
    if (dom.current instanceof HTMLElement) {
      dom.current.addEventListener(props.trigger, handelEvent);
    }
  }, []);
  return (
    <span ref={dom} className="custom-report">
      {props.children}
    </span>
  );
};

export default CusReport;

Components are used as follows:

 <CusReport tag="test" label="功能测试" trigger="click">
  <button>测试</button>
</CusReport>

This is more elegant, you don't need to modify the target element, just wrap the component outside the target element.

Summarize

This article introduces how to build front-end monitoring to collect behavior data, and divide the data into two categories: general data and specific data . At the same time, various ways of reporting data are also introduced, and different ways can be selected for different scenarios.

The data part only introduces the basic fields for realizing functions, and can be added according to your own business needs in actual situations.

Many friends have left comments on whether this front-end monitoring can be open-sourced. It must be open-sourced, but I am still working on a lot of content. I will post a version when it is basically perfected. Thank you for your attention.

This series of articles is as follows, welcome to connect with one click:

• Why can't the front end be without a monitoring system?
• Don't be confused about the construction steps of front-end monitoring!
• How to collect abnormal data when building front-end monitoring?

For more dry goods, please pay attention to the success of the official account programmer , and add the author WeChat to the front-end group.