Caching & PWA Practice

1. Background

From the previous article "Front-end Animation Implementation and Principle Analysis" , we perform animation performance analysis from Performance, and optimize animation according to Performance analysis. But, the front end is more than just implementing smooth animations. ToB projects often deal with data saving and rendering. For example, in the development, in order to improve the user experience, we encountered some scenarios, in fact, we are using some means to optimize the performance.

Select drop-down: do front-end paging display → avoid rendering the data at one time and cause the browser to freeze;
indexedDB: store data → the next time the user enters, save the state of the last edit...

That inevitably leads to thinking, we think from the cache and data storage, how to optimize?

2. HTTP Cache

what is it?

Http cache actually means that the browser saves all the resources obtained through HTTP.
It is a behavior of the browser to store network resources locally.

Where is the requested resource?

6.8kB + 200 Status code: no cache hit, actual request, get resources from the server;
memory cache: The resource is cached in the memory, and the server will not be requested. Generally, the resource has been loaded and cached in the memory. When the page is closed, the resource will be released by the memory;
disk cache: The resource is cached in the disk, the server will not be requested, but the resource will not be released when the page is closed;
304 status code: request the server and find that the resource has not been changed, after returning 304, the resource is taken out from the local;
service worker: Application-level storage means;

HTTP cache classification

1. Strong cache

When the browser loads a resource, it will first judge whether the strong cache is hit according to the information in the header of the locally cached resource. If it hits, it will not send the request like the server, but read the resource directly from the cache.
Strong caching can be achieved by setting HTTP Header:

http1.0 → Expires: The response header contains the date/time after which the response expires.
http1.1 → Cache-Control:max-age=: Set the maximum period of cache storage, beyond which the cache is considered to be expired (in seconds). In contrast to Expires the time is relative to the request

💡 Cache-control

cache-control: max-age=3600 : indicates relative time
cache-control:no-cache → can be stored in the local cache, but the cache cannot make it available to the client until the origin server has verified the freshness
cache-control: no-store → prohibits the cache from copying the response, that is, the real data is not cached locally;
catch-control: public → can be cached by all users (shared by multiple users), including terminals, CDNs, etc.
cache-control: private → private cache
2. Negotiate cache
1. When the browser's request for a resource does not hit the strong cache, it will send a request to the server to verify whether the negotiated cache is hit. If the negotiated cache hits, the request will return an http status of 304 and the string of Not Modified will be displayed;
2. The negotiation cache is managed by [last-Modified, if-Modified-Since] and [ETag, if-None-Match].

"Last-Modified, If-Modified-Since"

last-Modified: Indicates the date when the local file was last modified. The browser will include if-Modified-since (also the value of Last-Modified returned last time) in the request header, and the server will compare this value with the time when the resource was modified. Match, if the time is inconsistent, the server will return a new resource and update the Last-modified value and return it to the browser as a response header. If the time is the same, it means that the resource has not been updated, the server will return a 304 status, and the browser will read the resource from the local cache after getting the response status code.

" ETag, If-None-Match"

In http 1.1, the server sets the response header cache flag through Etag. Etags are generated by the server.

On the first request, the server will return the resource and the ETag to the browser, and the browser will cache both in the local cache.

In the second request, the browser will put the value of ETag in the If-None-Match request header to access the server. After the server receives the request, it will compare the file ID in the server with the ID sent by the browser. If If they are different, the server will return the updated resource and the new Etag. If they are the same, the server will return a 304 status code and the browser will read the cache.

💡 Think about why there is a Last-Modified pair, and Etag is needed to identify whether it has expired and hit the negotiation cache

The file is changed periodically, but the content of the file has not changed, only the modification time has been changed. At this time, the client does not want the client to think that the file has been modified and obtain it again.
If the file is modified frequently, such as N times in 1s, If-Modified-Since cannot determine the modification, it will cause the file to be modified but the acquired resources are still old, which will cause problems.
Some servers cannot accurately get the last modification time of a file, causing resource acquisition problems.

⚠️If Etag and Last-Modified exist at the same time, the server will check ETag first, then check Last-Modified, and finally return 304 or 200.

HTTP Caching Practices

Test environment: Use Koa to build a node service to test how to hit strong cache or negotiate cache

When index.js is not configured with any cache configuration, no matter how you refresh chrome, there is no cache mechanism;

Note⚠️: Before starting the experiment, you need to uncheck Disable cache in the network panel. This option means to disable browser cache, and the browser request will bring the Cache-Control: no-cache and Pragma: no-cache header information.

1. Hit strong cache

 app.use(async (ctx) => {
    // ctx.body = 'hello Koa'
    const url = ctx.request.url;
    if(url === '/'){
        // 访问跟路径返回 index.html
        ctx.set('Content-type', 'text/html');
        ctx.body = await parseStatic('./index.html')
    }else {
        ctx.set('Content-Type', parseMime(url))
        ctx.set('Expires', new Date(Date.now() + 10000).toUTCString()) // 实验1
        ctx.set('Cache-Control','max-age=20') // 实验2
        ctx.body = await parseStatic(path.relative('/', url))
    }
})

app.listen(3000, () => {
    console.log('starting at port 3000')
})

2. Hit the negotiation cache

 /**
         * 命中协商缓存
         * 设置 Last-Modified, If-Modified-Since
         */
         ctx.set('cache-control', 'no-cache'); // 关闭强缓存
         const isModifiedSince = ctx.request.header['if-modified-since'];
         const fileStat = await getFileStat(filePath);
         if(isModifiedSince === fileStat.mtime.toGMTString()){
             ctx.status = 304
         }else {
             ctx.set('Last-Modified', fileStat.mtime.toGMTString())
         }  
         ctx.body = await parseStatic(path.relative('/', url))

        /**
         * 命中协商缓存
         * 设置 Etag, If-None-Match
         */
         ctx.set('cache-control', 'no-cache');
         const ifNoneMatch = ctx.request.headers['if-none-match'];
         const fileBuffer = await parseStatic(filePath);
         const hash = crypto.createHash('md5');
         hash.update(fileBuffer);
         const etag = `"${hash.digest('hex')}"`
         if (ifNoneMatch === etag) {
            ctx.status = 304
          } else {
            ctx.set('Etag', etag)
            ctx.body = fileBuffer
          }
    }

3. Browser cache

1. Cookies

MDN definition: It is a small piece of data sent by the server to the user's browser and reported locally. It will be carried and sent to the server the next time the browser wants to unify the server to send a request again.
Application scenarios:
- Session state management [user login status, shopping cart, game score or other information that needs to be recorded]
- Personalization settings (such as user-defined settings, themes, etc.)
- Browser behavior tracking (such as tracking and analyzing user behavior, etc.)
Cookie reading and writing:

 class Cookie {
    getCookie: (name) => {
        const reg = new RegExp('(^| )'+name+'=([^;]*)')
        let match = document.cookie.match(reg); //  [全量，空格，value，‘；’]
        if(match) {return decodeURI(match[2])}
    }
    setCookie:(key,value,days,domain) => {
        // username=John Smith; expires=Thu, 18 Dec 2043 12:00:00 GMT; path=/
      let d = new Date();
    d.setTime(d.getTime()+(days*24*60*60*1000));
    let expires = "; expires="+d.toGMTString();
        let domain = domain ? '; domain='+domain : '';
        document.cookie = name + '=' + value + expires + domain + '; path=/'
        
    }
    deleteCookie: (name: string, domain?: string, path?: string)=> {
        // 删除cookie，只需要将时间设置为过期时间，而无需删除cookie的value
        const d = new Date(0);
        domain = domain ? `; domain=${domain}` : '';
        path = path || '/';
        document.cookie =
            name + '=; expires=' + d.toUTCString() + domain + '; path=' + path;
    },
}

Existing problems: Due to the data stored through cookies, each request will be carried in the request header. For some data, there is no need to submit it to the backend, which will inevitably bring additional overhead.

2. WebStorage API

Browsers can store key-value pairs in a more intuitive way than using cookies

localStorage

Maintains a separate storage area for each given origin, and the data persists after the browser is closed and then reopened.

sessionStorage

Maintains a separate storage area for each given origin that is available during a page session (i.e. as long as the browser is open, including page reloads and restores).

3. indexedDB and webSQL

webSQL

The basic operation is basically the same as the actual database operation.
The final destination of the data is generally only used for temporary storage and business operation storage and caching of large websites. After the page is refreshed, the library does not exist. And it itself is similar to the concept of relational database.

indexedDB

With the continuous enhancement of browser functions, more and more websites have begun to consider storing a large amount of data on the client side, which can reduce the data obtained from the server and obtain data directly from the local. Existing browser data storage solutions are not suitable for storing large amounts of data;

IndexedDB is a local database provided by the browser, which allows to store a large amount of data, provides a search interface, and can also build indexes. These are not available in LocalStorage. In terms of database type, IndexedDB is not a relational database (does not support SQL query statements), and is closer to a NoSQL database.

Fourth, the application cache

Service Worker

Before mentioning Service Worker, you need to have some understanding of Web Worker;

webWorker : Web Workers are browser built-in threads so can be used to execute JavaScript code in a non-blocking event loop. js is a single thread, and can only complete one thing at a time. If a complex task occurs, the thread will be blocked, which will seriously affect the user experience. The role of Web Worker is to allow the main thread to create a worker thread and perform it at the same time as the main thread. The worker thread only needs to be responsible for the complex calculation, and then return the result to the main thread. The simple understanding is that the worker thread performs complex calculations and the page (main thread) ui is very smooth and will not be blocked.

A Service Worker is a virtual proxy between the browser and the network. It addresses how to properly cache future website resources and make them available offline.

Service Worker runs on a separate thread from the main page js thread and has no access to the DOM structure. Its API is non-blocking and can send and receive messages between different contexts.

Not only does it provide offline functionality, it can also do things like handle notifications, perform heavy computations on a separate thread, and more. Service Workers are very powerful because they can control network requests, modify network requests, and return cached custom or synthetic responses.

2. PWA

💡 PWA, the full name of Progressive web apps, is a progressive web application. The main role of PWA technology is to build cross-platform web applications and make them have the same user experience as native applications.
💡 The core of PWA: The most fundamental and most basic is the Service Worker and the Cache API used inside it. As long as Service Worker and Cache API are used, the caching of website pages, the interception of page requests, and the manipulation of page caches are realized.

Why use PWAs:

Problems with the traditional web:

Lack of direct entry, need to remember his domain name, or save it in the favorites, it is not convenient to find;
Depends on the network. As long as the client is disconnected from the network, the entire web is paralyzed and the client cannot use it;
Unable to push messages like Native APP.

Problems with traditional native apps:

Requires installation and download. Even if you just use a certain function of the APP, you need to download it in full;
High development costs, generally need to be compatible with Android and IOS systems;
Posting requires review;
The cost of updating is high...

The existence of PWA is to solve the troubles caused by the above problems:
Advantage:

Desktop entrance, easy to open;
Available offline, without over-reliance on the network;
Easy to install;
One-time development, no need to audit, all platforms are available;
Ability to push messages
Web App Manifest The Web App Manifest (Web Application Manifest) is, in a nutshell, a file that centrally writes page-related information and configuration in JSON.

 {
  "short_name": "User Mgmt",
  "name": "User Management",
  "icons": [
    {
      "src": "favicon.ico",
      "sizes": "64x64 32x32 24x24 16x16",
      "type": "image/x-icon"
    },
    {
      "src": "logo192.png",
      "type": "image/png",
      "sizes": "192x192"
    },
    {
      "src": "logo512.png",
      "type": "image/png",
      "sizes": "512x512"
    }
  ],
  "start_url": ".", // 调整网站的起始链接
  "display": "standalone", // 设定网站提示模式 ： standalone 表示隐藏浏览器的UI
  "theme_color": "#000000", // 设定网站每个页面的主题颜色
  "background_color": "#ffffff" // 設定背景顏色
}

ServiceWorker.js code

 /* eslint-disable no-restricted-globals */

// 确定哪些资源需要缓存
const CACHE_VERSION = 0;
const CACHE_NAME = 'cache_v' + CACHE_VERSION;
const CACHE_URL = [
  '/',
  'index.html',
  'favicon.ico',
  'serviceWorker.js',
  'static/js/bundle.js',
  'manifest.json',
  'users'
]
const preCache = () => {
  return caches
    .open(CACHE_NAME)
    .then((cache) => {
      return cache.addAll(CACHE_URL)
    })
}
const clearCache = () => {
  return caches.keys().then(keys => {
    keys.forEach(key => {
      if (key !== CACHE_NAME) {
        caches.delete(key)
      }
    })
  })
}
// 进行缓存
self.addEventListener('install', (event) => {
  event.waitUntil(
    preCache()
  )
})

// 删除旧的缓存
self.addEventListener('activated', (event) => {
  event.waitUntil(
    clearCache()
  )
})

console.log('hello, service wold');

self.addEventListener('fetch', (event) => {
  console.log('request:', event.request.url)
  event.respondWith(
    fetch(event.request).catch(() => { // 优先网络请求，如果失败，则从缓存中拿资源
      return caches.match(event.request)
    })
  )
})

PWA debugging

When offline, the cached resources are still obtained and displayed normally. It can be seen that the resources are cached by serviceWorker.

With developer tools:
chrome devtools : chrome://inspect/#service-workers , which shows the active and stored service workers on the current device

V. Summary and thinking

Refer to the excellent website:

Yuque: https://www.yuque.com/dashboard ;
PWA example: https://mdn.github.io/pwa-examples/js13kpwa/

Caching & PWA Practice