Preface
In the process of blog development, there is a need to solve, that is, in the SSR development environment, the server code is directly packaged into files through webpack (because it contains isomorphic code, that is, the server and the client share the front end Component code), write it to the disk, and then start the service after starting the packaged entry file. But I don't want to pack the files into the disk in the development environment. I want to pack them directly in the memory. This not only optimizes the speed, but also does not generate redundant files due to the development environment. There is also the processing of require by webpack, which will cause path mapping problems, including the problem of require variables. So I think that only component-related code will be compiled by webpack, and other irrelevant server-side codes will not be compiled by webpack.
But there is a problem that has remained unresolved, that is, how to introduce files in memory. Including how to import the associated files together after importing this file, such as require(module) , so I thought of the vue-server-renderer library that was used when doing ssr for vue. This is not directly outputting the file, but The file was pushed into memory. But he can get the file, and execute the file to get the result. So started this research journey.
achieve
vue-server-renderer 's talk about the implementation process of the project first, and then talk about how the 060a39030ab586 package solves this problem, so as to implement it in React.
|-- webpack
| |-- webpack.client.js // entry => clilent-main.js
| |-- webpack.server.js // entry => server-main.js
|-- client // 客户端代码
| |-- app.js
| |-- client-main.js // 客户端打包入口
| |-- server-main.js // server端打包代码入口
|-- server // server端代码
| |-- ssr.js // ssr启动入口client-main.js, the client packs a copy of the code, which is normal packaging, and packs out the corresponding files.import React, { useEffect, useState } from 'react' import ReactDom from 'react-dom' import App from './app' loadableReady(() => { ReactDom.hydrate( <Provider store={store}> <App /> </Provider>, document.getElementById('app') ) })server-main.js, because it is SSR, a corresponding js file needs to be packaged on the server side for ssr rendering. Here I intend to directly process the component-related data in this block and return the html. Then the server will directly import this file and get the html and return it to the front-end. This is the processing of my project. The official vue demo will be a little different. It will directly return the app instance (new Vue(...), and thenvue-server-rendererlibrary. Finally, it will also return the parsed html string. There will be a little difference here, The principle remains the same.// 返回一个函数，这样可以传入一些参数，用来传入服务端的一些数据 import { renderToString } from 'react-dom/server' export default async (context: IContext, options: RendererOptions = {}) => { // 获取组件数据 ... // 获取当前url对应的组件dom信息 const appHtml = renderToString( extractor.collectChunks( <Provider store={store}> <StaticRouter location={context.url} context={context as any}> <HelmetProvider context={helmetContext}> <App /> </HelmetProvider> </StaticRouter> </Provider> ) ) // 渲染模板 const html = renderToString( <HTML>{appHtml}</HTML> ) context.store = store return html }ssr.js, because these files are all saved in memory. So I need to parse the file in memory to getserver-main.js, execute it, and return html to the front end.// start方法是执行webpack的node端代码，用于把编译的文件打入内存中。 import { start } from '@root/scripts/setup' // 执行他，createBundleRenderer方法就是用来解析在server端打包的代码 start(app, ({ loadableStats, serverManifest, inputFileSystem }) => { renderer = createBundleRenderer({ loadableStats, serverManifest, inputFileSystem }) }) // 执行server-main.js中的函数并获取html const html = await renderer.renderToString(context) ctx.body = html
For the client, it is easy to say that by creating an html template, and then importing the resources corresponding to the current route (js, css, ..), when accessing, the browser can directly pull the resources (this is through @loadable/webpack-plugin , @loadable/server , @loadable/component for resources The loading and acquisition of, I won’t introduce too much here, the focus of this article is not on this).
The focus of this block is how to parse the server-main.js packaged code that needs to be referenced on the server side in memory.
Let's look at the official code of vue ssr: vue-hackernews-2.0
const VueSSRServerPlugin = require('vue-server-renderer/server-plugin')
module.exports = merge(base, {
target: 'node',
devtool: '#source-map',
entry: './src/server-main.js',
output: {
filename: 'server-bundle.js',
libraryTarget: 'commonjs2'
},
plugins: [
new VueSSRServerPlugin()
]
})
vue-server-renderer/server-plugin used above. The main function of this plug-in is what it does. In fact, it processes the resources in webpack and puts all the js resources in a json file.
The source code is as follows:
// webpack上自定义了一个vue-server-plugin插件
compiler.hooks.emit.tapAsync('vue-server-plugin', (compilation, cb) => {
// 获取所有资源
var stats = compilation.getStats().toJson();,
var entryName = Object.keys(stats.entrypoints)[0];
var entryInfo = stats.entrypoints[entryName];
// 不存在入口文件
if (!entryInfo) {
return cb()
}
var entryAssets = entryInfo.assets.filter(isJS);
// 入口具有多个js文件，只需一个就行： entry: './src/entry-server.js'
if (entryAssets.length > 1) {
throw new Error(
"Server-side bundle should have one single entry file. " +
"Avoid using CommonsChunkPlugin in the server config."
)
}
var entry = entryAssets[0];
if (!entry || typeof entry !== 'string') {
throw new Error(
("Entry \"" + entryName + "\" not found. Did you specify the correct entry option?")
)
}
var bundle = {
entry: entry,
files: {},
maps: {}
};
// 遍历所有资源
stats.assets.forEach(function (asset) {
// 是js资源，就存入bundle.files字段中。
if (isJS(asset.name)) {
bundle.files[asset.name] = compilation.assets[asset.name].source();
} else if (asset.name.match(/\.js\.map$/)) { // sourceMap文件，存入maps字段中，用来追踪错误
bundle.maps[asset.name.replace(/\.map$/, '')] = JSON.parse(compilation.assets[asset.name].source());
}
// 删除资源，因为js跟js.map已经存到bundle中了，需要的资源已经存起来了，别的没必要打包出来了。
delete compilation.assets[asset.name];
});
var json = JSON.stringify(bundle, null, 2);
var filename = this$1.options.filename; // => vue-ssr-server-bundle.json
// 把bundle存入assets中，那样assets中就只有vue-ssr-server-bundle.json这个json文件了，
/*
vue-ssr-server-bundle.json
{
entry: 'server-bundle.js',
files: [
'server-bundle.js': '...',
'1.server-bundle.js': '...',
],
maps: [
'server-bundle.js.map': '...',
'1.server-bundle.js.map': '...',
]
}
*/
compilation.assets[filename] = {
source: function () { return json; },
size: function () { return json.length; }
};
cb();
});The processing of this plug-in is also extremely simple, that is, the resource is intercepted and processed again. Generate a json file, and then it will be convenient for direct analysis and processing.
Then we look at the entry file of the node service to see how to get the html and parse it
const { createBundleRenderer } = require('vue-server-renderer')
// bundle: 读取vue-ssr-server-bundle.json中的数据，
/*
bundle => vue-ssr-server-bundle.json
{
entry: 'server-bundle.js',
files: [
'server-bundle.js': '...',
'1.server-bundle.js': '...',
],
maps: [
'server-bundle.js.map': '...',
'1.server-bundle.js.map': '...',
]
}
*/
renderer = createBundleRenderer(bundle, {
template: fs.readFileSync(templatePath, 'utf-8'), // html模板
// client端json文件，也存在于内存中，也是对webpack资源的拦截处理，这里不做多介绍，原理差不多。读取对应的资源放入html模板中，在client端进行二次渲染，绑定vue事件等等
clientManifest: readFile(devMiddleware.fileSystem, 'vue-ssr-client-manifest.json'),
runInNewContext: false // 在node沙盒中共用global对象，不创建新的
}))
const context = {
title: 'Vue HN 2.0', // default title
url: req.url
}
renderer.renderToString(context, (err, html) => {
if (err) {
return handleError(err)
}
res.send(html)
})By looking at the entry file started by the server-side project above, createBundleRenderer in renderToString used to directly return html, so I came to vue-server-renderer to see what was done in this library.
function createRenderer(ref) {
return {
renderToString: (app, context, cb) => {
// 解析app： app => new Vue(...),就是vue实例对象
// 这块就是对vue组件的编译解析，最后获取对应的html string
// 重点不在这，此处也不做过多介绍
const htmlString = new RenderContext({app, ...})
return cb(null, htmlString)
}
}
}
function createRenderer$1(options) {
return createRenderer({...options, ...rest})
}
function createBundleRendererCreator(createRenderer) {
return function createBundleRenderer(bundle, rendererOptions) {
entry = bundle.entry;
// 关联的js资源内容
files = bundle.files;
// sourceMap内容
// createSourceMapConsumers方法作用便是通过require('source-map')模块来追踪错误文件。因为我们都进行了资源拦截，所以这块也需要自己实现对错误的正确路径映射。
maps = createSourceMapConsumers(bundle.maps);
// 调用createRenderer方法获取renderer对象
var renderer = createRenderer(rendererOptions);
// 这块就是处理内存文件中的代码了，
// {files: ['entry.js': 'module.exports = a']}, 就是我读取entry.js文件中的内容，他是字符串, 然后node如何处理的,处理完之后得到结果。
// 下面这个方法进行详细说明
var run = createBundleRunner(
entry,
files,
basedir,
rendererOptions.runInNewContext
);
return {
renderToString: (context, cb) => {
// 执行run方法,就能获取我在server-main.js入口文件里面 返回的new Vue实例
run(context).then(app => {
renderer.renderToString(app, context, function (err, res) {
// 打印错误映射的正确文件路径
rewriteErrorTrace(err, maps);
// res: 解析好的html字符串
cb(err, res);
});
})
}
}
}
}
var createBundleRenderer = createBundleRendererCreator(createRenderer$1);
exports.createBundleRenderer = createBundleRenderer;- The above logic is also clear. The string code of the entry file is parsed
createBundleRunnerserver-main.jsentry file is a Promise function, and the Promise returnsnew Vue(), so the parsed result isnew Vueinstance of 060a39030abd56. new Vueinstance throughRenderContextand other instances to obtain the corresponding html string.- Correct the file path mapping for the error through the
source-map
In this way, the code in the file is executed in memory, html is returned, and the effect of ssr is achieved. The focus of this article is how to execute the string code of the entry file.
We come to the createBundleRunner method to see how it is implemented.
function createBundleRunner (entry, files, basedir, runInNewContext) {
var evaluate = compileModule(files, basedir, runInNewContext);
if (runInNewContext !== false && runInNewContext !== 'once') {
// 这块runInNewContext不传false 跟 once这两个选项的话，每次都会生成一个新的上下文环境，我们共用一个上下文global就行。所以这块就不考虑
} else {
var runner;
var initialContext;
return function (userContext) {
// void 0 === undefined, 因为undefined可被重新定义，void没法重新定义，所以用void 0 肯定是undefined
if ( userContext === void 0 ) userContext = {};
return new Promise(function (resolve) {
if (!runner) {
// runInNewContext： false, 所以这里上下文就是指的global
var sandbox = runInNewContext === 'once'
? createSandbox()
: global;
// 通过调用evaluate方法返回入口文件的函数。代码实现： evaluate = compileModule(files, basedir, runInNewContext)
// 去到compileModule方法看里面是如何实现的
/*
vue官方demo的server-main.js文件,返回的时一个Promise函数，所以runner就是这个函数。
export default context => {
return new Promise((resolve) => {
const { app } = createApp()
resolve(app)
})
}
*/
// 传入入口文件名，返回入口函数。
runner = evaluate(entry, sandbox);
}
// 执行promise返回 app，至此app就得到了。
resolve(runner(userContext));
});
}
}
}
// 这个方法返回了evaluateModule方法，也就是上面evaluate方法
// evaluate = function evaluateModule(filename, sandbox, evaluatedFiles) {}
function compileModule (files, basedir, runInNewContext) {
var compiledScripts = {};
// filename: 依赖的文件名，例如 server.bundle.js 或 server.bundle.js依赖的 1.server.bundle.js 文件
// 在通过vue-ssr-server-bundle.json中的files字段获取这个文件名对应的文件内容 类似："module.exports = 10"字符串
// 通过node的module模块来包裹这段代码，代码其实很简单粗暴，封装成了一个函数，传入我们熟知的commonjs规范中的require、exports等等变量
/*
Module.wrapper = [
'(function (exports, require, module, __filename, __dirname, process, global) { ',
'\n});'
];
Module.wrap = function(script) {
return Module.wrapper[0] + script + Module.wrapper[1];
};
结果：
function (exports, require, module, __filename, __dirname, process, global) {
module.exports = 10
}
*/
// 通过vm模块创建沙盒环境，来执行这段js代码。
function getCompiledScript (filename) {
if (compiledScripts[filename]) {
return compiledScripts[filename]
}
var code = files[filename];
var wrapper = require('module').wrap(code);
var script = new require('vm').Script(wrapper, {
filename: filename,
displayErrors: true
});
compiledScripts[filename] = script;
return script
}
function evaluateModule (filename, sandbox, evaluatedFiles) {
if ( evaluatedFiles === void 0 ) evaluatedFiles = {};
if (evaluatedFiles[filename]) {
return evaluatedFiles[filename]
}
// 获取这个执行这段代码的沙盒环境
var script = getCompiledScript(filename);
// 沙盒环境使用的上下文 runInThisContext => global
var compiledWrapper = runInNewContext === false
? script.runInThisContext()
: script.runInNewContext(sandbox);
var m = { exports: {}};
var r = function (file) {
file = path$1.posix.join('.', file);
// 当前js依赖的打包文件，存在，继续创建沙盒环境执行
if (files[file]) {
return evaluateModule(file, sandbox, evaluatedFiles)
} else {
return require(file)
}
};
// 执行函数代码。注意webpack要打包成commonjs规范的,不然这里就对不上了。
compiledWrapper.call(m.exports, m.exports, r, m);
// 获取返回值
var res = Object.prototype.hasOwnProperty.call(m.exports, 'default')
? m.exports.default
: m.exports;
evaluatedFiles[filename] = res;
// 返回结果
return res
}
return evaluateModule
}
In fact, there are not many implementations in the createBundleRunner Is to create a sandbox environment to execute the obtained code
The core idea of the whole logic is as follows
- Generate a json file by intercepting webpack assets, including all js file data
- Pass the entry file to the generated json file to extract the string code.
require('module').wrapthe string code into a functional string code through 060a39030abeea, commonjs specification- Create a sandbox environment through
require('vm')to execute this code and return the result. - If the entry file depends on other files, perform steps 2-4 again and replace the entry file with the dependent file. For example, routes are generally lazy loaded, so when accessing the specified route, webpack will also get it when it is packaged. This corresponding routing file depends on the entry file.
- The returned result obtained through the execution of the sandbox environment is
new Vueinstance object vue-hackernews-2.0 - Parse this
vueinstance, get the corresponding html string, put it into the html template, and finally return it to the front end.
In this way, the memory file is read and the corresponding html data is obtained. These codes are mainly executed through the vm module and the module In fact, the entire code of this block is relatively simple. There is no complicated logic.
Because the project is based on react and webpack5 , there will be some differences in the code processing, but the implementation scheme is basically the same.
In fact, when it comes to executing code, there is another method in js that can execute code, that is, the eval method. However eval method was used in require , the search was performed in the local module. I found that the files in the memory could not be searched require Therefore, the code executed by the vm module is still used, after all, the require method can be rewritten
The complete code of the project: GitHub repositoryI have created a new group to learn from each other. Whether you are a Xiaobai who is about to enter the pit, or a classmate who is halfway into the industry, I hope we can share and communicate together.
QQ group: 810018802, click to join
| QQ group | the public |
|---|---|
Front end miscellaneous group  | Winter Melon Bookstore  |
**粗体** _斜体_ [链接](http://example.com) `代码` - 列表 > 引用。你还可以使用@来通知其他用户。