"8 minutes to learn the principle of Vue.js": 1. The template string is compiled into an abstract syntax tree AST

Vue.js does not have any mysterious magic, template rendering, virtual DOM diff are also implemented line by line based on API.

This article will take a few minutes to explain the principle of rendering Vue.js 2.0 <template> to HTML DOM in chapters.

If you have any questions, welcome to communicate through comments~~

Objectives of this section

Compile the string template of `template` into an abstract syntax tree AST;

Complete example: DEMO - "Learn Vue.js Principles in 8 Minutes": 1. Compile template strings into abstract syntax trees AST - JSBin

3670fe62c282c35ae6c293bb85c8f6c

Extremely simple core logic

The core logic of implementing "string template <template> compiled to render() function" is extremely simple, with only 2 parts:

String.prototype.match()

First, use the .match() method to extract the keywords in the string, such as the tag name div , the variable corresponding to the Mustache tag msg and so on.

It can be explained with 1 line of code:

 '<div>{{msg}}</div>'.match(/\{\{((?:.|\r?\n)+?)\}\}/)
// ["{{msg}}", "msg"]

This example uses the /\{\{((?:.|\r?\n)+?)\}\}/ regular expression to extract '<div>{{msg}}</div>' Mustache标签 .

You don't need to understand the meaning of this regex for the time being, you can use it.

If you want to understand the regularity, you can try the regular online debugging tool: Vue.js start tag regular , which can visually check the above regular expression

After obtaining the name of the variable corresponding to the "msg" tag, we can splicing out the _vm.msg required for rendering the DOM later.

That is, extract ---3c87d81cc30ba4c8093675b646866980--- from our declared instance new Vue({data() {return {msg: 'hi'}}}) msg: 'hi' and render it as a DOM node.

Iterate over string and delete traversed result

Secondly, because <template> is essentially a string with a large number of HTML tags, usually with a long content, in order to get all the tags and attributes in it, we need to traverse.

The implementation method is also very simple. Use while(html) to loop and continuously html.match() to extract the information in the template. (html variable is template string)

Each time you extract a paragraph, use html = html.substring(n) to delete n a character that has been traversed.

Until the html string is empty, it means that we have traversed and extracted all template .

 html = `<div`   // 仅遍历一遍，提取开始标签

const advance = (n) => {
    html = html.substring(n)
}

while (html) {
    match = html.match(/^<([a-zA-Z_]*)/)
    // ["<div", "div"]
    if (match) {
        advance(match[0].length)
        // html = '' 跳出循环         
    }
}

After understanding these 2 parts of logic, I understand the principle of the string template template compiled as render() function, it's that simple!

Specific steps

0. Based on `class` syntax package

We use JS class syntax to simply encapsulate and modularize the code, specifically to declare 3 classes:

 // 将 Vue 实例的字符串模板 template 编译为 AST
class HTMLParser {}

// 基于 AST 生成渲染函数；
class VueCompiler {
    HTMLParser = new HTMLParser()
}

// 基于渲染函数生成虚拟节点和真实 DOM
class Vue {
    compiler = new VueCompiler()
}

Q: Why is AST generated? What's wrong with compiling the template string template directly into the real DOM?
A: No problem, it can be achieved technically,
Vue.js and many compilers use AST as the intermediate state of compilation. Personal understanding is to do "transform" during the compilation process.
For example, v-if the attribute is converted into JS if-else judgment, with AST as an intermediate state, it is helpful to realize more convenient v-if to if-else

1. Start traversing `template` string template

Based on our above mentioned while() and html = html.substring(n) ,

We can implement a set of logic that parses the template string and deletes the parsed parts until all parsing is complete.

It's simple, just a few lines of code,

We add a parseHTML(html, options) method for class HTMLParser e5607aff0654a3c88463b15f3907b28b---:

 parseHTML(html, options) {
  const advance = (n) => {
    html = html.substring(n)
  }

  while(html) {
    const startTag = parseStartTag()  // TODO 下一步实现 parseStartTag
    if (startTag) {
      advance(startTag[0].length)
      continue
    }
  }
}

html parameter is the template: '<div>{{msg}}</div>', attribute in the initialization Vue instance,

In the process of traversing html, every time we parse out a keyword, we call advance() { html.substring(n) } , delete this part of the corresponding string,

In the example, we call parseStartTag() (the next step to implement) to parse out the corresponding string of the start tag <div> , then delete the 5 characters from html <div> .

2. Parse the start tag `<div>`

Next, let's implement parseStartTag() and parse out the start tag in the string <div> .

It is also very simple, just use html.match(regularExp) , we can find the corresponding regular expression startTagOpen from html-parser.js in the Vue.js source code .

The /^<([a-zA-Z_]*)/ of the source code is very complicated, because it needs to be compatible with various labels of the production environment, we will not consider it for the time being.

Online debugging start label regular

Use this regular call to html.match() to get the array ['<div', 'div'] .

After extracting the required information, call advance(start[0].length) to delete the characters that have been traversed.

 const parseStartTag = () => {
  let start = html.match(this.startTagOpen);
  // ['<div', 'div']
  if (start) {
    const match = {
      tagName: start[1],
      attrs: [],
    }
    advance(start[0].length)
  }
}

Note that startTagOpen only matches the part of the start tag '<div' , and a regular match is needed to find the end symbol of the start tag > .

After finding the end symbol, also delete the corresponding traversed part.

 const end = html.match(this.startTagClose)
debugger
if (end) {
  advance(end[0].length)
}
return match

After the two parts are combined, the <div> start tag in the template string can be completely traversed and parsed.

3. Parse the text content `{{msg}}`

Next, we extract the text content in the template string {{msg}} , it is still 字符串遍历 && 正则匹配

Continue to add the while loop:

 while (html) {
  debugger
  // 顺序对逻辑有影响，startTag 要先于 text，endTag 要先于 startTag
  const startTag = parseStartTag()
  if (startTag) {
    handleStartTag(startTag)
    continue
  }

  let text
  let textEnd = html.indexOf('<')
  if (textEnd >= 0) {
    text = html.substring(0, textEnd)
  }

  if (text) {
    advance(text.length)
  }
}

Because the parsed part is deleted and each part has a parsing order, we only need to detect the position of the next < tag to get the ending subscript of the text content in html: html.indexOf('<') .

Then you can get the full text content {{msg}} : text = html.substring(0, textEnd) .

Finally, don't forget to delete the text content that has been traversed: advance(text.length)

4. Parse closing tag `</div>`

At this point, there is only html string left '</div>' , we continue to traverse && regular parsing:

 while (html) {
  debugger
  // 顺序对逻辑有影响，startTag 要先于 text，endTag 要先于 startTag
  let endTagMatch = html.match(this.endTag)
  if (endTagMatch) {
    advance(endTagMatch[0].length)
    continue
  }
}

We don't need to extract information from the closed tag for now, so just traverse, match, and delete it.

Analysis completed summary

So far, we have basically implemented class HTMLParser {} , and used regular parsing to extract 3 parts of information in the template string:

Start tag: '<div', '>'
Text content: '{{msg}}'
Closing tags: '</div>'

For the complete code of this part, you can visit DEMO - "Learn Vue.js Principles in 8 Minutes": 1. The template string is compiled into an abstract syntax tree AST - JSBin to view.

But in order to get the AST, we also need to do some simple splicing based on this information.

5. Initialize the root node of the abstract syntax tree AST

We continue to refer to the implementation of splicing AST in the Vue.js source code

Improve class VueCompiler , add HTMLParser = new HTMLParser() instance, and parse(template) method.

 class VueCompiler {
  HTMLParser = new HTMLParser()

  constructor() {}

  parse(template) {}
}

What is AST?

Don't need to understand the obscure concepts first. In the implementation of Vue.js, AST is an ordinary JS object , which records attributes such as tag name, parent element, and child element:

 createASTElement (tag, parent) {
  return { type: 1, tag, parent, children: [] }
}

We also added the createASTElement method to class VueCompiler .

And add the call of this.HTMLParser.parseHTML() in the ---11a48cdfa355a84e42e6510eecfbc582 parse method

 parse(template) {
  const _this = this
  let root
  let currentParent

  this.HTMLParser.parseHTML(template, {
    start(tag) {},
    chars (text) {},
  })

  debugger
  return root
}

start(tag) {} is the callback for extracting the start tag corresponding to the AST node,

It accepts one parameter tag and calls _this.createASTElement(tag, currentParent) to generate the AST node.

  start(tag) {
    let element = _this.createASTElement(tag, currentParent)

    if (!root) {
      root = element
    }
    currentParent = element
  },

Call the start(tag) method in the position of class HTMLParser in parseHTML(html, options) :

   const handleStartTag = (match) => {
    if (options.start) {
      options.start(match.tagName)
    }
  }

  while(html) {
    const startTag = parseStartTag()
    if (startTag) {
      handleStartTag(startTag)
      continue
    }
  }

When we get ---024adb99c21a8f799d363b801e46543f--- through parseStartTag() {tagName: 'div'} , we pass it to options.start(match.tagName) , thus generating the root node of AST:

 // root
'{"type":1,"tag":"div","children":[]}'

We save the root node to the root variable, which will eventually return a reference to the entire AST.

6. Add child nodes to AST

In addition to the root node, we also need to continue to add child nodes to the AST tree: text content nodes

Still in the form of a callback ( options.char(text) ), extract the information required by the text content node,

Improve the VueCompiler.parse() chars(text)

 chars(text) {
  debugger
  const res = parseText(text)
  const child = {
    type: 2,
    expression: res.expression,
    tokens: res.tokens,
    text
  }
  if (currentParent) {
    currentParent.children.push(child)
  }
},

In the loop of parseHTML(html, options) b450629c494f7bcd65a41e3386c13c57--- add options.chars(text) call:

 while (html) {
  // ...省略其他标签的解析
  let text
  let textEnd = html.indexOf('<')
  // ...

  if (options.chars && text) {
    options.chars(text)
  }
}

`Mustache标签` syntax for parsing text content

options.chars(text) The received text value is a string '{{msg}}' , we also need to remove {{}} , and get msg

Still use the familiar regex match:

   const defaultTagRE = /\{\{((?:.|\r?\n)+?)\}\}/
  function parseText(text) {
    let tokens = []
    let rawTokens = []
    const match = defaultTagRE.exec(text)

    const exp = match[1]
    tokens.push(("_s(" + exp + ")"))
    rawTokens.push({ '@binding': exp })

    return {
      expression: tokens.join('+'),
      tokens: rawTokens
    }
  }

The result will be:

 {
  expression: "_s(msg)",
  tokens: {
    @binding: "msg"
  }
}

It is not necessary to understand the specific meaning of expression, tokens and its content for the time being. We will introduce it in detail later in the runtime stage.

7. Traversal `template` string complete, return AST

Complete example: DEMO - "Learn Vue.js Principles in 8 Minutes": 1. Compile template strings into abstract syntax trees AST - JSBin

After the above steps, we parse the template string and get such an object:

 // root ===
{
    "type": 1,
    "tag": "div",
    "children": [
        {
            "type": 2,
            "expression": "_s(msg)",
            "tokens": [
                {
                    "@binding": "msg"
                }
            ],
            "text": "{{msg}}"
        }
    ]
}

This is the AST of Vue.js, the implementation is so simple, the code in the example comes directly from the source code of Vue.js ( compiler part )

Later, we will generate render() function based on AST, and finally render the real DOM.

<hr/>

"8 minutes to learn Vue.js principle" series, a total of 5 parts:

The update is in full swing, welcome to communicate~ Welcome to urge update~

"8 minutes to learn the principle of Vue.js": 1. The template string is compiled into an abstract syntax tree AST