Do you only use front-end data to bury the SDK?

Preface

I believe that many people have not taken the time to understand it because they have not been exposed to the content related to data burying in the project. They always feel that this is another aspect that they cannot cover yet. However, the data burying itself is not difficult to understand, but it is very difficult to understand. It is difficult to do well. This article will start from the two core aspects of understanding the data burying SDK and designing the front-end data burying SDK , and talk about the things about front-end data burying.

Get to know the data burrowing SDK

The full name of SDK is 

Software Development Kit , which is generally a collection of development tools used by software engineers to create application software for specific software packages, software frameworks, hardware platforms, operating systems, etc.

Why do we need front-end data embedding?

As for the product itself, we need to pay attention to the following aspects:

  • What operations do users mainly perform in the product, how long do they stay, and how many times do they visit?
  • What is the proportion of user click-through rates? Are there certain functional designs that are ineffective for users?
  • Is the user’s core usage process smooth and whether the page feedback is normal and friendly?
  • What potential user features may need to be updated?

In general, the core of data burying is to collect data (With data, you can do whatever you want), only by analyzing data can we better evaluate the quality and importance of the entire project (Data is king), and can provide direction for product optimization ( data-driven products ).

What aspects should be considered when burying front-end data?

The core of data burying is data collection, and the content related to data is nothing more than the following:

  • Data is generated based on applications, because without applications there would be no relevant data.
  • The application itself must provide display, collection, and operation content, which is based on the platform. For example, the website is based on the browser platform.
  • To have an application or a platform, there must be users, because the application itself is to provide users with useful functions to solve certain existing problems.
  • For developers, an application is code, and the quality of the code running can also determine the quality of the application, and explicit quality is reflected in errors or warnings.

To sum up, data burying should actually consider three core aspects: user behavior, error warnings, and page performance .

user behavior

User behavior is a series of operations performed in a web application. However, there are many kinds of user operations, and it is impossible to record them all. Generally, the following user behaviors need to be recorded:

  • Number of times a user views a page, PV (Page View)
    • Each time a user visits a page on the website , it is recorded 1asPV
  • Number of users browsing the page, UV (Unique visitor)
    • For users who normally access the page through the network, usually one computer client or one user account is a visitor. Generally, 24hmultiple visits by the same client or user account will only be recorded as 1one UV. The calculation strategy depends on the specific situation.
  • Number of button clicks by users
    • The above two can be considered as yes 自动式触发埋点, and the number of clicks on the button belongs to yes 互动式触发埋点, which is convenient for understanding the usage of this function button.
    Error warningErrors generated by the code running on the page may interrupt the user’s core operation process. In order to prevent a large number of users from being affected, we need to obtain error data from the production environment so that developers can make timely repairs.

Generally speaking, errors in the code will include the following categories:

  • Global errors , i.e. uncaught errors
  • Local errors , i.e. try...catch、promise.then、promise.catcherrors caught via etc.
  • Interface request errorAPI , that is , an error when making a request and receiving a response in the secondary encapsulation request
  • Component-level errors , i.e. Vue/Reacterrors that occur when using a component

Page performance

In fact, page performance is also a point that needs to be considered and optimized in front-end performance optimization. After all, if a website always suffers from problems such as white screen, interactive lag, and long loading time of page resources , it will definitely not be able to retain users, especially users. The real environments are different, such as Windows x、MACOS、Android、iOSetc., which requires more statistics and collection of relevant data to facilitate centralized optimization and improve user experience.

Regarding content related to page performance indicators, how to do the previous front-end performance optimization (Part 1) – Getting straight to the point is mentioned in the article, here is a rough summary:

  • first draw ( First Paint，FP)
    • After the rendering process confirms that it wants to render the current response resource, the rendering process will first create a blank page. This point in time when the blank page is created is usually called First Paint, referred to asFP
    • The so-called white screen time actually refers to the time from when this blank page is created to when the browser starts rendering non-blank content, such as when the page background changes, etc.
  • First content draw ( First Contentful Paint，FCP)
    • The point in time when the user sees some “content” elements being drawn on the page is different from the white screen. It can be 文本the first draw, or SVGthe first appearance, or the first draw, etc., that is, when the first pixelCanvas is drawn on the page. , this time point is called ,abbreviationFirst Content PaintFCP
  • First screen time/maximum content drawing ( Largest Contentful Paint, LCP)
    • LCPIt is a new performance metric that LCPfocuses on user experience. Compared with existing metrics, it is easier to understand and reason. When the first screen content is completely drawn, this time point is called Largest Content Paint, for short.LCP
    • Maximum content drawing should 2.5sbe completed within
  • First input delay ( First Input Delay, FID)
    • FIDWhat is measured is the time from when the user interacts on the page for the first time ( clicking a link , clicking a button , or customizing jsan event based on ) to the time the browser actually starts processing this event.
    • First input delay should 100msbe completed within
  • Cumulative layout offset ( Cumulative Layout Shift, CLS)
    • CLSis to measure visual stability in order to provide a good user experience
    • The cumulative layout offset should be kept at  0.1or less
  • First byte arrival time ( Time to First Byte，TTFB)
    • It refers to the time when the browser starts to receive the server response data ( background processing time + redirection time ). It is an important indicator that reflects the response speed of the server.
    • TTFBIf the time exceeds 500ms, users will feel an obvious wait when opening the web page.

After understanding why we need to bury front-end data and all aspects of the statistical data required for front-end data burying, we need to design our own front-end data burying SDK .

Design front-end data embedding point SDK

Here we only consider the core content of data burying points, so it will not be covered in a comprehensive manner, and it is impossible to design it comprehensively at the beginning. As long as the core functions are guaranteed, then expansion based on the core can be done.

Determine options and data content

The unique identifier of the application — options.AppId

As a general tool set, the data burying point SDK can be used by multiple systems, which means that it is necessary to ensure the uniqueness of each application. Generally speaking, when initializing the SDK , an access method is required. The ID of the current application provided .

So where does this ID come from? Generate it casually? Generally speaking, you need to go through the following steps:

  • Generate a unique AppId for the current application on the corresponding monitoring system
  • Pass it in as one of the configuration items when the corresponding application accesses the SDK.

In fact, the request content will also be involved url, which is mainly used to send to the corresponding monitoring system. Therefore, optionsthe core content is simply designed as follows:

{
   appId : '' , // The unique identifier of the current application 
  baseUrl : '' , // The address where the data is sent 
}

Data sending format—data

Since there are many types of data that need to be collected, it is best to define a more general data format to facilitate more friendly data collection.

Here is a brief definition of the data format, which is roughly as follows. The format varies depending on the demand scenario:

{
   appId : '' , // The unique identifier of the current application 
  type : 'action' | 'performance' | 'network' | 'error' , // Different data types 
  pageUrl : '' , // Page address 
  apiUrl : '' , / /Interface address 
  userId : '' , //Current user id 
  userName : '' , //Current user name 
  time : '' , //Trigger recording time 
  data : {}, //Interface response result|Performance indicator|Error object | User operation related information 
}

Determine how data is sent

If you want to ask what is the most basic function to be achieved by front-end embedding, it must be the ability to send data . Otherwise, even if there are applications, users, and data, they can only be saved locally and cannot be sent to the corresponding monitoring system, which means that Unable to collect and count (Data is free).

So what are the ways to send data? For this problem, it is much easier to translate data sending into request sending . The question then becomes what are the request sending methods?

Generally, they include the following types (including but not limited to):

  • XMLHttpRequest
  • fetch
  • formformaction
  • Requests based on element srcattributes
    • imgLabeledsrc
    • scriptLabeledsrc
    • Navigator.sendBeacon()

The last one is chosen here because Navigator.sendBeacon()it is specifically used to  asynchronously  send  statistical data  to the server through HTTP POST , and at the same time it can avoid some problems of sending analysis data with traditional technology.Web

Some problems with traditional technology for sending statistical data can be viewed directly through 传送门. Due to the limited space of the article, no additional explanation will be given.

SDK core code

Here we only consider the minimalist case. The designed SDK code content is relatively simple, so you can directly enter the code:

let  SDK = null  // EasyAgentSDK instance object 
const  QUEUE = [] // Task queue 
cosnt NOOP = ( v ) => v

//Page performance indicators through web-vitals 
const  reportWebVitals = ( onPerfEntry ) => {
   if (onPerfEntry && onPerfEntry instanceof  Function ) {
     import ( 'web-vitals' ). then ( ( { getCLS, getFID, getFCP, getLCP, getTTFB } ) => {
       getCLS (onPerfEntry) // Layout offset 
      getFID (onPerfEntry) // First input delay time 
      getFCP (onPerfEntry) // First content rendering time 
      getLCP (onPerfEntry) // First maximum content rendering time 
      getTTFB (onPerfEntry) / / First byte arrival time
    })
  }
}

export  default  class  EasyAgentSDK {
  appId = '' 
  baseUrl = '' 
  timeOnPage = 0
  config = {}
  onPageShow = null 
  onPagesHide = null
  
  constructor ( options = {} ) {
     if ( SDK ) return

    SDK = this 
    this . appId = options. appId 
    this . baseUrl = options. baseUrl || window . location . origin 
    this . onPageShow = options. onPageShow || NOOP 
    this . onPagesHide = options. onPagesHide || NOOP

    //Initialize listening for page changes 
    this . listenPage ()
  }
  
  //Set config 
  setConfig ( congfig ){
     this . config = congfig
  }

  // Refresh the task queue 
  flushQueue () {
     Promise . resolve () . then ( () => {
       QUEUE . forEach ( ( fn ) =>  fn ())
       QUEUE . length = 0 ;
    })
  }

  // Listen for page changes 
  listenPage () {
     let pageShowTime = 0

    window . addEventListener ( 'pageshow' , () => {
      pageShowTime = performance. now ()
      
       // Page performance indicator reporting 
      reportWebVitals ( ( data ) => {
         this . performanceReport ({ data })
      })
      
      //Execute onPageShow 
      this . onPageShow ();
    })

    window . addEventListener ( 'pagehide' , () => {
       // Record the time the user stays on the page 
      this . timeOnPage = performance. now () - pageShowTime
      
      // Execute onPageShow before refreshing the queue 
      this . onPageShow ();

      // Refresh the task queue 
      this . flushQueue ()
    })
  }

  // Json to FormData 
  json2FormData ( data ){
     const formData = new  FormData ()

    Object . keys (data) . forEach ( key => {
      formData.append ( key , data[key])
    });

    return formData
  }

  // Custom reporting type 
  report ( config ) {
     QUEUE . push ( () => {
       const formData = json2FormData ({
        ... this . config ,
        ...config,
        time : new  Date (). toLocaleString (),
         appId : this . appId ,
         pageUrl : window . location . href ,
      });
      navigator. sendBeacon ( ` ${ this .baseUrl} ${config.url || '' } ` , formData)
    })
  }

  // User behavior reporting 
  actionReport ( config ) {
     this . report ({
      ...config,
      type : 'action' ,
    })
  }

  // Network status reporting 
  networkReport ( config ) {
     this . report ({
      ...config,
      type : 'network' ,
    })
  }

  // Page performance indicator reporting 
  performanceReport ( config ) {
     this . report ({
      ...config,
      type : 'performance' ,
    })
  }

  // Error warning reporting 
  errorReport ( config ) {
     this . report ({
      ...config,
      type : 'error' ,
    })
  }
}

Report user behavior

Statistics of PV and UV – automatically trigger buried points

PV and UV have been introduced above. Essentially, these two data statistics can be obtained in a reporting type of data actiontransmission. It mainly depends on what rules the monitoring system analyzes and counts the data according to. This is SDKinternal Monitored pageshow / pagehidetwo events on the page:

  • PV/UV related data and page performance related data pageshowcan be reported inwindow . SDK = new EasyAgentSDK ({ appId : ‘application_id’ , baseUrl : ‘//aegis.example.com/collect’ , onPageShow () { window . SDK . actionReport ({ data : {} // Other necessary information to pass }) } }); window . SDK . setConfig ({ userId : UserInfo . userId , // current user id userName : UserInfo . userName , // current user name });
  • It pagehideis mainly used to calculate the time the user stays on the page timeOnPageand refresh the task queue.

Count user clicks on buttons—interactive triggering of hidden points

Suppose we want to record data on the number of times certain buttons are used. We can documentlisten to clickevents on and use event bubbling so that we don’t need to intrude clickevents on different buttons, for example:

const  TargetElementFilter = [ 'export_btn' ]

const  findTarget = ( filters ) => {
  return filters. find ( ( filter ) =>  TargetElementFilter . find ( ( v ) => filter === v)));
}

document . addEventListener ( 'click' , ( e ) => {
   const { id, className, outerHTML } = e. target 
  const isTarget = findTarget ([id, className])

  if (isTarget) {
     SDK . actionReport ({
       data : {
        ID,
        className,
        outerHTML
      }, // Other necessary information to be passed
    })
  }
})

Report page performance

Content related to page performance is SDKautomatically triggered and should not be manually accessed by users. In the above implementation, we report data through and pageshowin the event , and here we choose to launch to obtain and page performance Specific data related to indicators, the corresponding codes are:reportWebVitalsperformanceReportGoogleweb-vitals

//Page performance indicators through web-vitals 
const  reportWebVitals = ( onPerfEntry ) => {
   if (onPerfEntry && onPerfEntry instanceof  Function ) {
     import ( 'web-vitals' ). then ( ( { getCLS, getFID, getFCP, getLCP, getTTFB } ) => {
       getCLS (onPerfEntry) // Layout offset 
      getFID (onPerfEntry) // First input delay time 
      getFCP (onPerfEntry) // First content rendering time 
      getLCP (onPerfEntry) // First maximum content rendering time 
      getTTFB (onPerfEntry) / / First byte arrival time
    })
  }
}

The data obtained is roughly as follows:

Report error warning

global error

Global errors, that is, uncaught errors, can be window.onerrorcaptured through events, and then error data is reported, roughly as follows:

window . addEventListener ( 'error' , ( reason ) => {
     const { filename, message, error } = reason;

    window . SDK . errorReport ({
         data : {
            filename,
            message,
            error
        }
    });
})

local error

Local errors, that is,  try...catch、promise.then、promise.catch errors caught by etc., are roughly used as follows:

 try {
     throw  new  Error ( 'error for test' )
  } catch (error) {
     window . SDK . errorReport ({
       data : {
        error,
      },
    })
  }


  Promise . reject ( new  Error ( 'Promise reject for test' ))
  . then (
     () => {},
     ( reason ) => {
       window . SDK . errorReport ({
         data : {
             error : reason
        }
    });
    },
  )
  
  Promise . reject ( new  Error ( 'Promise reject for test' ))
  . catch (
     ( reason ) => {
       window . SDK . errorReport ({
         data : {
             error : reason
        }
    });
    },
  )

Interface request error

Interface request errors, that is,  API errors when making requests and receiving responses in secondary encapsulation requests. To facilitate the example axioshere , we can report the corresponding errors in the second callback parameter of its request interception and response interception. The data information is roughly as follows:

// Create an axios instance 
const service = axios. create ({
  baseURL, // api's base_url 
  timeout : 60000 , // request timeout 
  responseType : reqConf. responseType ,
});

// Request interception 
service. interceptors . request . use (
   ( config ) => {
    ...
    return config;
  },
  ( error ) => {
     window . SDK . errorReport ({
       apiUrl : config. url ,
       data : {
        error,
      },
    })
  },
);

// Response interception 
service. interceptors . response . use (
   ( config: any ) => {
    ...
    return config;
  },
  ( error: any ) => {
     window . SDK . errorReport ({
       apiUrl : config. url ,
       data : {
        error,
      },
    })

    return error.response.data ;​​​
  },
);

Component level errors

Component-level errors, that is,  Vue / React errors that occur when using framework components, can be captured and reported using the error capture methods mentioned in their official documentation.

  • Vuein errorHandleris used to specify a global handler for uncaught errors thrown within the application
// App.vue
onMounted(()=>{
  throw new Error('error in onMounted')
});

// main.ts
const app = createApp(App)

app.config.errorHandler = (error, instance, info) => {
    window.SDK.errorReport({
        data: {
            instance,
            info,
            error
        }
    });
}
  • ReactErrorBoundaryError boundary related getDerivedStateFromErrorand componentDidCatchhooks in
// 定义错误边界组件
class ErrorBoundary extends React.Component {
  constructor(props) {
    super(props);
    this.state = { hasError: false };
  }

  static getDerivedStateFromError(error) {    
      // 更新 state 使下一次渲染能够显示降级后的 UI    
      return { hasError: true };  
  }
  componentDidCatch(error, info) {    
      // 可以将错误日志上报给服务器    
      window.SDK.errorReport({
        data: {
            info,
            error
        }
    });
  }
  render() {
    if (this.state.hasError) {      
        // 自定义降级后的 UI 并渲染      、
        return <h1>Something went wrong.</h1>;    
    }
    return this.props.children; 
  }
}

// 使用错误边界组件
<ErrorBoundary>
  <MyWidget />
</ErrorBoundary>

at last

Now we know a few things about front-end data burying SDK . The above example may make you think it looks relatively simple, but it is not that easy to do a good job in data burying. For example, you need to consider your SDK data . The time of sending, the number of times to send, whether it is necessary to integrate certain data information and only send it once, how to avoid network congestion, etc.

By JavaScript