NestJS builds front-end routing service

background

Usually, in order to better manage and maintain the project, the project is generally divided into business categories, such as commodities, orders, members, etc., resulting in many front-end projects with different business responsibilities ( SPA , single-page application). Assuming there is a demand now, all front-end projects need to access the Shence buried point Web JS SDK . If the static page index.html of each front-end project is used to introduce Web JS SDK , then each project needs to be redeployed after it is introduced, and it needs to be re-deployed in the future. When replacing the third-party buried point SDK , the previous steps need to be repeated, which is quite troublesome. And if you add a routing and forwarding layer in front of all front-end projects, which is a bit like a front-end gateway, intercepting responses and introducing Web JS SDK , then a lot of simple and error-prone workload will be avoided.

Front-end routing service

When a browser accesses a single-page application, it actually obtains the application's index.html file for parsing and rendering, and the gateway forwards the request to the front-end routing service (here, the front-end routing service is the node service, which is registered and discovered using k8s ), and the front-end routing service routes to the origin site according to the request. (Similar to Alibaba Cloud Object Storage Service OSS , the front-end project deployment above) Get the corresponding index.html

During the process of loading and parsing html , the browser will request static resources of css and js from CDN when it encounters the tags link and scipt CDN

Now you may have a question, why the html file is not obtained from CDN , but obtained directly from the source site, isn't it slower? Don't worry, I'll talk about it later

Qiniu Cloud simulates actual project deployment

Front-end projects will be deployed to object storage services, such as Alibaba Cloud Object Storage Service OSS , Huawei Cloud Object Storage Service OBS , here I use Qiniu Cloud Object Storage Service to simulate the actual deployment environment

1. Create a storage space, create a three-level static resource directory www/cassmall/inquiry , and then upload a index.html simulate actual project deployment

2. Configure the origin site domain name and CDN domain name for the storage space (the actual configuration needs to be filed with the domain name first), request index.html to use the origin site domain name, request js , css , img and other static resources to use the CDN domain name

Explain why you can obtain index.html from the origin site instead of the CDN domain name? hypothesis by CDN get index.html , when the first deployment of a single page application, assuming that browser to access http://localhost:3000/mall/inquiry/#/xxx , CDN not on index.html the source station to pull index.html , then CDN cached copy; when to index.html was amended a second time Deploy (deploy to the origin site), the browser still visits http://localhost:3000/mall/inquiry/#/xxx and finds that CDN (old) already exists on index.html , and returns it directly to the browser instead of returning the latest index.html of the origin index.html . CDN . If you directly use the origin site domain name to request index.html , then every time you get the latest index.html .

In fact, it is also possible to obtain index.html through the CDN domain name, but you need to set CDN cache configuration so that it does not cache the files with the html suffix.

In addition, for static resources such as js , css , img , and video , we want pages to be loaded quickly, so we use CDN to speed up the acquisition. js and css may be changed frequently, but after the build, they will generate hash and rename the file according to the content. If the file is changed, its hash will also change, and the CDN cache will not be hit when the request is made, and it will return to the source; if the file has not changed, its hash will not change, it will hit CDN cache. img and video do not change very frequently. If you need to change, you can rename and upload to prevent the same name from hitting CDN cache.

Nest Build front-end routing service

project creation

First make sure you have installed Node.js , the Node.js installation will come with npx and a npm package to run the program. Make sure you have Node.js (>= 10.13.0, except v13) installed on your OS. To create a new Nest.js application, run the following command on the terminal:

npm i -g @nestjs/cli  // 全局安装Nest
nest new web-node-router-serve  // 创建项目

After executing the create project, the following files will be initialized, and you will be asked if there is a way to manage dependencies:

If you have installed yarn , you can choose yarn , it can be faster, npm will be slower to install in China

Then follow the prompts to run the project:

Project structure

Enter the project and see that the directory structure should look like this:

Here is a brief description of these core files:

src
├── app.controller.spec.ts
├── app.controller.ts
├── app.module.ts
├── app.service.ts
├── main.ts

app.controller.ts	Basic Controller for a single route
app.controller.spec.ts	Unit tests for controllers
app.module.ts	The root module of the application (Module)
app.service.ts	A basic service (Service) with a single method
main.ts	The entry file of the application, which uses the core function NestFactory to create an instance of the Nest application.

main.ts file contains an asynchronous function that bootstraps the :

import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';

async function bootstrap() {
  const app = await NestFactory.create(AppModule);
  await app.listen(3000);
}
bootstrap();

To create an instance of the Nest application, we used the NestFactory core class. NestFactory exposes some static methods for creating instances of applications. Among them, the create() method returns an application object that implements the INestApplication interface. In the main.ts example above, we only started the HTTP listener, which enables the application to listen for HTTP requests.

Application entry file

Let's adjust the entry file main.ts , the port can be set by command input:

import { INestApplication } from '@nestjs/common';
import { NestFactory } from '@nestjs/core';
import { AppModule } from './app.module';

const PORT = parseInt(process.env.PORT, 10) || 3334; // 端口

async function bootstrap() {
  const app = await NestFactory.create<INestApplication>(AppModule);
  await app.listen(PORT);
}
bootstrap();

configuration variable

Some configuration variables that the project needs to set are used globally and managed using the config package. Install config :

yarn add config

Create a new config directory in the root directory, add default.js , development.js , production.js , and add the following configuration:

• default.js configures the mapping relationship between the index.html request path and the actual storage path at the origin site
• development.js and production.js configure the domain name of the origin site in different environments

// default.js
module.exports = {
  ROUTES: [
    {
      cdnRoot: 'www/cassmall/inquiry', // 源站静态资源的存储路径
      url: ['/cassmall/inquiry'], // 请求路径
    },
    {
      cdnRoot: 'www/admin/vip',
      url: ['/admin/vip'],
    },
  ],
};

// development.js
module.exports = {
  OSS_BASE_URL: 'http://r67b3sscj.hn-bkt.clouddn.com/', // 开发环境源站域名
};

// production.js
module.exports = {
  OSS_BASE_URL: 'http://r737i21yz.hn-bkt.clouddn.com/', // 生产环境源站域名
};

Talk about config.get() obtain configuration variables rule: If NODE_ENV is empty, development.js , if not development.js , using default.js ; if NODE_ENV is not empty, then the config directory to find the corresponding file, if the file is not found then use default.js in Content; if the configuration item is not found in the specified file, go to default.js to find it.

Create a route controller

Use the scaffolding command to initialize the project, after setting the configuration variables, now let's create the routing control layer

// app.controller.ts
import {
  Controller,
  Get,
  Header,
  HttpException,
  HttpStatus,
  Req,
} from '@nestjs/common';
import { AppService } from './app.service';
import { Request } from 'express';
import config from 'config'; // esm方式引入cjs规范模块

type Route = { gitRepo: string; cdnRoot: string; url: string[] };
const routes = config.get('ROUTES'); // 获取路由配置变量
const routeMap: { [key: string]: Route } = {};
routes.forEach((route) => {
  for (const url of route.url) {
    routeMap[url] = route;
  }
});

@Controller()
export class AppController {
  constructor(private readonly appService: AppService) {}

  @Get(Object.keys(routeMap))
  @Header('X-UA-Compatible', 'IE=edge,chrome=1')
  async route(@Req() request: Request): Promise<string> {
    const path = request.path.replace(/\/$/g, '');
    const route = routeMap[request.path];
    if (!route) {
      // 抛出异常，Nest内置异常层会自动处理，生成JSON响应
      throw new HttpException(
        '没有找到当前url对应的路由',
        HttpStatus.NOT_FOUND,
      );
    }
    // 获取请求路径对应的静态页面
    return this.appService.fetchIndexHtml(route.cdnRoot);
  }
}

esm introduces cjs

The third-party module config is a module of the cjs specification. Before using the esm method to introduce the cjs , you need to add the configuration to the tsconfig.json :

{
  "compilerOptions": {
      "allowSyntheticDefaultImports": true, // ESM导出没有设置default，被引入时不报错
    "esModuleInterop": true, // 允许使用ESM带入CJS
  }  
}

Of course, you can directly use the cjs specification to import const config = require('config') or change it to import * as config from 'config' , otherwise the following error will be reported at runtime:

because esm imports cjs , esm has the concept of default , but cjs does not. resulting in imported config value of undefined

Any exported variable of appears to be an attribute on the object module.exports in cjs , and the export of esm of default is only an attribute of cjs on module.exports.default . After setting esModuleInterop：true; , tsc will add default attribute to module.exports when compiling

// before
import config from 'config';

console.log(config);
// after
"use strict";

var _config = _interopRequireDefault(require("config"));

function _interopRequireDefault(obj) { return obj && obj.__esModule ? obj : { "default": obj }; }

console.log(_config["default"]);

To understand this part of modular processing, you can refer to [tsc, babel, webpack's processing of module import and export ]( https://segmentfault.com/a/1190000041384332 )

@Get accepts an array of routing paths

@Get() HTTP request method decorator can accept a routing path array type, telling the controller which routing path requests can be handled

/**
 * Route handler (method) Decorator. Routes HTTP GET requests to the specified path.
 *
 * @see [Routing](https://docs.nestjs.com/controllers#routing)
 *
 * @publicApi
 */
export declare const Get: (path?: string | string[]) => MethodDecorator;

exception handling

When there is no corresponding route in the routing configuration, an exception is thrown. If there is no custom exception interception processing, the built-in exception layer of Nest will automatically process it and generate a JSON response.

const path = request.path.replace(/\/$/g, '');
const route = routeMap[request.path];
if (!route) {
  throw new HttpException(
    '没有找到当前url对应的路由',
    HttpStatus.NOT_FOUND,
  );
}

// 异常将会被Nest自动处理，生成下面JSON响应
{
  "statusCode": 404,
  "message": "没有找到当前url对应的路由"
}

Nest comes with a built-in exception layer that handles all unhandled exceptions in the application. When your application code does not handle an exception, the layer will catch the exception and then automatically send the appropriate user-friendly response.

Out of the box, this operation is performed by the built-in global exception filter , which handles exceptions of type HttpException (and its subclasses). When exception does not recognize (neither HttpException nor the class HttpException inherits from), the built-in exception filter produces the following default JSON response:

{
  "statusCode": 500,
  "message": "Internal server error"
}

Nest Automatic wrapping request handler returns

You can see that the above request handler directly returns html string, and the page request gets the 200 status code and the text/html type of response body. What's going on? In fact, Nest uses two options of and to manipulate the concept of response:

Standard (recommended)

Using this built-in method, when a request handler returns JavaScript object or array, it is automatically serialized to JSON . However, when it returns a JavaScript primitive type (eg, string , ), Nest will just send the value without attempting to serialize it. This makes response handling simple: just return the value and Nest handles the rest. Also, the status code of the response is always 200 by default, except for POST requests that use 201 . We can easily change this behavior by adding a decorator at the handler level (see status code ). number boolean @HttpCode(...)

library specific

We can use the library-specific (eg, Express ) response object , which can be injected using a decorator (eg, findAll(@Res() response) ) in the @Res() method handler signature. With this approach, you can use the native response handling methods exposed by this object. For example, with Express , you can use something like response.status(200).send() .

The service layer gets the static page

The control layer receives the page request and calls the service layer method to obtain the static page

// app.service.ts
import { HttpException, HttpStatus, Injectable } from '@nestjs/common';
import config from 'config';
import { IncomingHttpHeaders } from 'http';
import rp from 'request-promise';

interface CacheItem {
  etag: string;
  html: string;
}

interface HttpError<E> extends Error {
  result?: E;
}

interface HttpClientRes<T, E> {
  err?: HttpError<E>;
  statusCode?: number;
  result?: T;
  headers?: IncomingHttpHeaders;
}

@Injectable()
export class AppService {
  // 缓存
  private cache: { [url: string]: CacheItem | undefined } = {};

  async fetchIndexHtml(cdnRoot: string): Promise<string> {
    // 源站静态页面存储路径
    const ossUrl = `${config.get('OSS_BASE_URL')}${cdnRoot}/index.html`;
    const cacheItem = this.cache[ossUrl];
    // 请求options
    const options = {
      uri: ossUrl,
      resolveWithFullResponse: true, // 设置获取完整的响应，当值为false时，响应体只有body，拿不到响应体中的headers
      headers: {
        'If-None-Match': cacheItem && cacheItem.etag,
      },
    };

    // 响应
    const httpRes: HttpClientRes<any, any> = {};

    try {
      const response = await rp(options).promise();
      const { statusCode, headers, body } = response;
      httpRes.statusCode = statusCode;
      httpRes.headers = headers;
      if (statusCode < 300) {
        httpRes.result = body;
      } else {
        const err: HttpError<any> = new Error(
          `Request: 请求失败，${response.statusCode}`,
        );
        err.name = 'StatusCodeError';
        err.result = body;
        httpRes.err = err;
      }
    } catch (err) {
      httpRes.statusCode = err.statusCode; // 对于 GET 和 HEAD 方法来说，当验证失败的时候（有相同的Etag），服务器端必须返回响应码 304 （Not Modified，未改变）
      httpRes.err = err;
    }
    if (httpRes.statusCode === HttpStatus.OK) {
      // 文件有变化，更新缓存，并返回最新文件
      const finalHtml = this.htmlPostProcess(httpRes.result);
      const etag = httpRes.headers.etag;
      this.cache[ossUrl] = {
        etag,
        html: finalHtml,
      };
      return finalHtml;
    } else if (httpRes.statusCode === HttpStatus.NOT_MODIFIED) {
      // 文件没有变化，返回缓存文件
      return this.cache[ossUrl].html;
    } else {
      if (!this.cache[ossUrl]) {
        throw new HttpException(
          `不能正常获取页面 ${cdnRoot}`,
          HttpStatus.NOT_FOUND,
        );
      }
    }
    // 兜底
    return this.cache[ossUrl].html;
  }

  // 预处理
  htmlPostProcess(html: string) {
    return html;
  }
}

The service layer requests static resources

The above uses the request-promise package to send the server ajax request, which depends on request . Both packages need to be installed:

yarn add request-promise request

After requesting index.html , the server will cache it. There are two main points:

• For the GET and HEAD methods, when the verification fails (with the same Etag ), the server (origin site) must return the response code 304 ( Not Modified , unchanged), and will not return html string response body. If the routing service does not cache, it can only return 404 to the browser; if the routing service is cached, it can return the cached index.html to the browser to prevent users from being unable to use it.
• If the origin site crashes, we still have the cache of the routing service as a cover, which can reduce the impact on user usage to a certain extent.

Static resource preprocessing

Now go back to the core part of building the front-end routing service. After obtaining the index.html of the single-page application deployed on the origin site through the front-end routing service, we can preprocess it according to actual needs, and then return it to the browser.

For example, at the beginning, we talked about the unified introduction of Shence to all single-page projects. Web JS SDK

// 预处理
htmlPostProcess(html: string) {
  const $ = cheerio.load(html);
  $('head').append(
    `<script type="text/javascript" src="https://mstatic.cassmall.com/assets/sensors/cassSensors.js"></script>`,
  );
  return $.html();
}

In addition, after the single-page application is built, the generated css and js will generally be injected into index.html with a relative reference path, such as:

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Document</title>
  <link href="./static/css/22.64074466.chunk.css" rel="stylesheet"/>
  <script src="./static/js/22.66c89a53.chunk.js"></script>
</head>
<body>
  <div id="root"></div>
</body>
</html>

Browser access https://ec-alpha.casstime.com/mall/inquiry#/xxx , pulling the distal end routing services originating station www/cassmall/inquiry/ directory index.html returned to the browser, the browser parses index.html found link , script reference, corresponding to the request will go css , js static resources, and link , script reference path They are all relative paths, relative to the path accessed by the browser. The actual requested paths are https://ec-alpha.casstime.com/mall/inquiry/static/css/22.64074466.chunk.css and https://ec-alpha.casstime.com/mall/inquiry/static/js/22.66c89a53.chunk.js . This request path is wrong (404). The static resources of css and js should be obtained CDN CDN Therefore, it is necessary to change the relative path to the absolute path obtained from CDN in the preprocessing function

<!-- 预处理后 -->
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Document</title>
  <!-- <link href="./static/css/22.64074466.chunk.css" rel="stylesheet"/> -->
  <link href="https://mstatic.cassmall.com/www/cassmall/inquiry/static/css/22.64074466.chunk.css" rel="stylesheet"/>
  <!-- <script src="./static/js/22.66c89a53.chunk.js"></script> -->
  <script src="https://mstatic.cassmall.com/www/cassmall/inquiry/static/js/22.66c89a53.chunk.js"></script>
</head>
<body>
  <div id="root"></div>
</body>
</html>

// app.utils.ts
export function joinUrlPath(baseUrl: string, ...paths: string[]): string {
  const urlObj = url.parse(baseUrl);
  urlObj.pathname = path.posix.join(urlObj.pathname, ...paths);
  const searchIdx = urlObj.pathname.indexOf('?');
  if (searchIdx > -1) {
    urlObj.search = urlObj.pathname.slice(searchIdx + 1);
    urlObj.pathname = urlObj.pathname.slice(0, searchIdx);
  }
  return url.format(urlObj);
}

export function isFullUrl(url: string) {
  return /^https?:|^\/\//.test(url);
}

// 预处理
htmlPostProcess(html: string, cdnRoot: string) {
  const $ = cheerio.load(html);
  const cdn = 'https://mstatic.cassmall.com'; // CDN域名
  const baseUrl = joinUrlPath(cdn, cdnRoot, '/'); // 拼接路径
  
  // 替换link相对路径引用
  $('link').each((index: number, ele: cheerio.TagElement) => {
    let href = ele.attribs['href'];
    if (href && !isFullUrl(href)) {
      href = joinUrlPath(baseUrl, href);
      ele.attribs['href'] = href;
    }
  });
  // 替换script相对路径引用
  $('script').each((index: number, ele: cheerio.TagElement) => {
    let src = ele.attribs['src'];
    if (src && !isFullUrl(src)) {
      src = joinUrlPath(baseUrl, src);
      ele.attribs['src'] = src;
    }
  });
  
  // 添加神策埋点Web JS SDK
  $('head').append(
    `<script type="text/javascript" src="https://mstatic.cassmall.com/assets/sensors/cassSensors.js"></script>`,
  );
  
  return $.html();
}

Custom log middleware

The log system needs to monitor all routing requests and should be set to global middleware. Use the INestApplication method provided by use() to define the global middleware

Execute the following command to generate the middleware file:

/** generate|g [options] <schematic> [name] [path] */
nest g mi logger middleware

Directory Structure

src
├── middleware
        ├── logger.middleware.spec.ts
        ├── logger.middleware.ts
├── app.controller.spec.ts
├── app.controller.ts
├── app.module.ts
├── app.service.ts
├── main.ts

bunyan is a simple and efficient 0620df27a7caf4 log Node.js for JSON , which is used here to build a log system. Create src/utils/logger.ts file and configure the initialization log instance individually:

// src/utils/logger.ts
import Logger, { LogLevelString } from 'bunyan';

// eslint-disable-next-line @typescript-eslint/no-var-requires
const pkg = require(`${process.cwd()}/package.json`);
const level = (process.env.LOG_LEVEL || 'info').toLowerCase() as LogLevelString;

const logger = Logger.createLogger({
  name: pkg.name,
  level,
});

export default logger;

Then, introduce a log instance in the middleware to help print the log:

// logger.middleware.ts
import { Injectable, NestMiddleware } from '@nestjs/common';
import { Request, Response, NextFunction } from 'express';
import logger from '../utils/logger';

@Injectable()
export class LoggerMiddleware implements NestMiddleware {
  use(req: Request, res: Response, next: NextFunction) {
    next();
    logger.info(
      {
        type: 'access',
        method: req.method,
        url: req.url,
        userAgent: req.headers['user-agent'],
        statusCode: res.statusCode,
      },
      `%s %s %d`,
      req.method,
      req.url,
      res.statusCode,
    );
  }
}

register middleware

// src/main.ts
app.use(new LoggerMiddleware().use);

Modify the startup command:

"scripts": {
    "start": "nest start“, // 改动前
    "start": "nest start | bunyan -L" // 改动后
}

Log printing effect:

Project address: https://github.com/Revelation2019/web-node-router-serve