The underlying principle and simple implementation of Mustache

Used vue all know that we can use in the template {{xx}} to render data the property, this syntax is called Mustache interpolation expressions, are easy to use, but also the hearts have a question, how does it do it? Next, let us find out!

1. Use regular to achieve

For example, there is such a template character

let tempStr2 = '我是一名{{develpoer}}，我在学习{{knowledge}}知识！';

Now you need to {{xxx}} string with data, then you can use regular to achieve

let tempStr2 = '我是一名{{develpoer}}，我在学习{{knowledge}}知识！';
let data = {
  develpoer: 'web前端程序猿',
  knowledge: 'Mustache插值语法'
};
let resultStr = tempStr2.replace(/{{(\w+)}}/g, function (matched, $1){
  // {{develpoer}} develpoer
  // {{knowledge}} knowledge
  console.log(matched, $1);
  return data[$1];
});
// 结果： 我是一名web前端程序猿，我在学习Mustache插值语法知识！
console.log('结果：', resultStr);

The disadvantage of using regular is that only simple interpolation syntax can be realized. A little more complicated functions such as loop, if judgment etc. cannot be realized.

2. The underlying idea of Mustache: tokens idea

let tempStr = `
  <ul>
    {{#students}}
        <li>
            <dl>
                <dt>{{name}}</dt>
                {{#hobbys}}
                    <dd>{{.}}</dd>
                {{/hobbys}}
            </dl>
        </li>
    {{/students}}
  </ul>
`;

When encountering such a template string, according to our previous programming thinking, most people must think about how to get the content between {{#students}} and {{/students}}. It is impossible to achieve with regular rules. Yes, thinking about this string in a daze for a long time still to no avail.

What if we the contents of this string in 1612dda8d1153a? For example, {{xxx}} is divided into one category, and ordinary strings except {{xxx}} are divided into one category, and they are stored in an array, such as:

This is the idea of tokens . We can do things if we get such an array. We don't have the final say on how to splice the data.

3. Disassemble and classify template strings

Idea (here assume that the delimiter is a pair of {{ }} ):

1. variables in the template string or use traverse, if the judgment place must be wrapped with {{}}
2. All ordinary strings are on {{ , so you can {{ the position of 0612dda8d1165f, and then intercept them
3. {{ has been intercepted, and now the template string becomes {{xxx}}<li>... , then how to get xxx now?
4. New idea-use string interception (don't think about regularization anymore~). The normal string in front of {{ has been {{ can also be intercepted. After intercepting {{ , the template string becomes xxx}}<li>...
5. xxx}}<li>... This string looks like the original template string, but {{ becomes }} , then we can do the same as step 2, find }} , and then intercept
6. After intercepting xxx , the string becomes }}<li>... , then we }} , and then go back to step 2, and loop until there is no string to intercept.

code implementation:

/**
 * 模板字符串扫描器
 * 用于扫描分隔符{{}}左右两边的普通字符串，以及取得{{}}中间的内容。（当然分隔符不一定是{{}}）
 */
class Scanner{
  constructor (templateStr) {
    this.templateStr = templateStr;
    this.pos = 0; // 查找字符串的指针位置
    this.tail = templateStr; // 模板字符串的尾巴
  }

  /**
   * 扫瞄模板字符串，跳过遇到的第一个匹配的分割符
   * @param delimiterReg
   * @returns {undefined}
   */
  scan(delimiterReg){
    if(this.tail){
      let matched = this.tail.match(delimiterReg);
      if(!matched){
        return;
      }
      if(matched.index != 0){ // 分隔符的位置必须在字符串开头才能进行后移操作，否则会错乱
        return;
      }
      let delimiterLength = matched[0].length;
      this.pos += delimiterLength; // 指针位置需加上分隔符的长度
      this.tail = this.tail.substr(delimiterLength);
      // console.log(this);
    }
  }

  /**
   * 扫瞄模板字符串，直到遇到第一个匹配的分隔符，并返回第一个分隔符(delimiterReg)之前的字符串
   * 如：
   *    var str = '我是一名{{develpoer}}，我在学习{{knowledge}}知识！';
   *    第一次运行：scanUtil(/{{/) => '我是一名'
   *    第二次运行：scanUtil(/{{/) => '我在学习'
   * @param delimiterReg 分割符正则
   * @returns {string}
   */
  scanUtil(delimiterReg){
    // 查找第一个分隔符所在的位置
    let index = this.tail.search(delimiterReg);
    let matched = '';
    switch (index){
      case -1: // 没有找到，如果没有找到则说明后面没有使用mustache语法，那么把所有的tail都返回
        matched = this.tail;
        this.tail = '';
        break;
      case 0: // 分隔符在开始位置，则不做任何处理
        break;
      default:
        /*
          如果找到了第一个分隔符的位置，则截取第一个分割符位置前的字符串，设置尾巴为找到的分隔符及其后面的字符串，并更新指针位置
         */
        matched = this.tail.substring(0, index);
        this.tail = this.tail.substring(index);
    }
    this.pos += matched.length;
    // console.log(this);
    return matched;
  }

  /**
   * 判断是否已经查找到字符串结尾了
   * @returns {boolean}
   */
  eos(){
    return this.pos >= this.templateStr.length;
  }
}

export { Scanner };

use:

import {Scanner} from './Scanner';

let tempStr = `
  <ul>
    {{#students}}
        <li>
            <dl>
                <dt>{{name}}</dt>
                {{#hobbys}}
                    <dd>{{.}}</dd>
                {{/hobbys}}
            </dl>
        </li>
    {{/students}}
  </ul>
`;
let startDeli = /{{/; // 开始分割符
let endDeli = /}}/; // 结束分割符

let scanner = new Scanner(tempStr);
console.log(scanner.scanUtil(startDeli)); // 获取 {{ 前面的普通字符串
scanner.scan(startDeli); // 跳过 {{ 分隔符

console.log(scanner.scanUtil(endDeli)); // 获取 }} 前面的字符串
scanner.scan(endDeli); // 跳过 }} 分隔符

console.log('---------------------------------------------');

console.log(scanner.scanUtil(startDeli)); // 获取 {{ 前面的普通字符串
scanner.scan(startDeli); // 跳过 {{ 分隔符

console.log(scanner.scanUtil(endDeli)); // 获取 }} 前面的字符串
scanner.scan(endDeli); // 跳过 }} 分隔符

result:


4. Convert the string template into an array of tokens

The previous Scanner can parse the string, and now we only need to assemble the template string.
Code

import {Scanner} from '../Scanner';

/**
 * 将模板字符串转换成token
 * @param templateStr 模板字符串
 * @param delimiters 分割符，它的值为一个长度为2的正则表达式数组
 * @returns {*[]}
 */
export function parseTemplateToTokens(templateStr, delimiters  = [/{{/, /}}/]){
  let [startDelimiter, endDelimiter] = delimiters;
  let tokens = [];
  if(!templateStr){
    return tokens;
  }
  let scanner = new Scanner(templateStr);

  while (!scanner.eos()){
    // 获取开始分隔符前面的字符串
    let beforeStartDelimiterStr = scanner.scanUtil(startDelimiter);
    if(beforeStartDelimiterStr.length > 0){
      tokens.push(['text', beforeStartDelimiterStr]);
      // console.log(beforeStartDelimiterStr);
    }
    // 跳过开始分隔符
    scanner.scan(startDelimiter);
    // 获取开始分隔符与结束分隔符之间的字符串
    let afterEndDelimiterStr = scanner.scanUtil(endDelimiter);
    if(afterEndDelimiterStr.length == 0){
      continue;
    }
    if(afterEndDelimiterStr.charAt(0) == '#'){
      tokens.push(['#', afterEndDelimiterStr.substr(1)]);
    }else if(afterEndDelimiterStr.charAt(0) == '/'){
      tokens.push(['/', afterEndDelimiterStr.substr(1)]);
    }else {
      tokens.push(['name', afterEndDelimiterStr]);
    }
    // 跳过结束分隔符
    scanner.scan(endDelimiter);
  }

  return tokens;
}

use:

import {parseTemplateToTokens} from './parseTemplateToTokens';
let tempStr = `
  <ul>
    {{#students}}
        <li>
            <dl>
                <dt>{{name}}</dt>
                {{#hobbys}}
                    <dd>{{.}}</dd>
                {{/hobbys}}
            </dl>
        </li>
    {{/students}}
  </ul>
`;
let delimiters = [/{{/, /}}/];
var tokens = parseTemplateToTokens(templateStr, delimiters);
console.log(tokens);

result:


5. Assemble the tokens again

There is a nested structure in the template string we used earlier, and the previously assembled tokens are a one-dimensional array. It is obviously impossible to use a one-dimensional array to render a template string with a loop structure. Even if it is possible, the code will be difficult to understand. .
At this point, we need to assemble the one-dimensional array again. This time we will assemble it into a nested structure, and the one-dimensional array encapsulated earlier is also eligible.
code:

/**
 * 将平铺的tokens数组转换成嵌套结构的tokens数组
 * @param tokens 一维tokens数组
 * @returns {*[]}
 */
export function nestsToken(tokens){
  var resultTokens = []; // 结果集
  var stack = []; // 栈数组
  var collector = resultTokens; // 结果收集器

  tokens.forEach(token => {
    let tokenFirst = token[0];
    switch (tokenFirst){
      case '#':
        // 遇到#号就将当前token推入进栈数组中
        stack.push(token);
        collector.push(token);
        token[2] = [];
        // 并将结果收集器设置为刚入栈的token的子集
        collector = token[2];
        break;
      case '/':
        // 遇到 / 就将栈数组中最新入栈的那个移除掉
        stack.pop();
        // 并将结果收集器设置为栈数组中栈顶那个token的子集，或者是最终的结构集
        collector = stack.length > 0 ? stack[stack.length - 1][2] : resultTokens;
        break;
      default:
        // 如果不是#、/则直接将当前这个token添加进结果集中
        collector.push(token);
    }
  });

  return resultTokens;
}

result of the call:

After this step, there is nothing particularly difficult. With such a structure, it is easy to combine data.

6. Render the template

The following code is my simple implementation:
code:

import {lookup} from './lookup';

/**
 * 根据tokens将模板字符串渲染成html
 * @param tokens
 * @param datas 数据
 * @returns {string}
 */
function renderTemplate(tokens, datas){
  var resultStr = '';
  tokens.forEach(tokenItem => {
    var type = tokenItem[0];
    var tokenValue = tokenItem[1];
    switch (type){
      case 'text': // 普通字符串，直接拼接即可
        resultStr += tokenValue;
        break;
      case 'name': // 访问对象属性
        // lookup是一个用来以字符串的形式动态的访问对象上深层的属性的方法，如：lookup({a: {b: {c: 100}}}, 'a.b.c')、lookup({a: {b: {c: 100}}}, 'a.b');
        resultStr += lookup(datas, tokenValue);
        break;
      case '#':
        let valueReverse = false;
        if(tokenValue.charAt(0) == '!'){ // 如果第一个字符是!，则说明是在使用if判断做取反操作
          tokenValue = tokenValue.substr(1);
          valueReverse = true;
        }
        let val = datas[tokenValue];
        resultStr += parseArray(tokenItem, valueReverse ? !val : val, datas);
        break;
    }
  });
  return resultStr;
}

/**
 * 解析字符串模板中的循环
 * @param token token
 * @param datas 当前模板中循环所需的数据数据
 * @param parentData 上一级的数据
 * @returns {string}
 */
function parseArray(token, datas, parentData){
  // console.log('parseArray datas', datas);
  if(!Array.isArray(datas)){ // 如果数据的值不是数组，则当做if判断来处理
    let flag = !!datas;
    // 如果值为真，则渲染模板，否则直接返回空
    return flag ? renderTemplate(token[2], parentData) : '';
  }
  var resStr = '';
  datas.forEach(dataItem => {
    // console.log('dataItem', dataItem);
    let nextData;
    if(({}).toString.call(dataItem) != '[object, Object]'){
      nextData = {
        ...dataItem,
        // 添加一个"."属性，主要是为了在模板中使用{{.}}语法时可以使用
        '.': dataItem
      }
    }else{
      nextData = {
        // 添加一个"."属性，主要是为了在模板中使用{{.}}语法时可以使用
        '.': dataItem
      };
    }

    resStr += renderTemplate(token[2], nextData);
  });
  return resStr;
}

export {renderTemplate, parseArray};

use:

import {parseTemplateToTokens} from './parseTemplateToTokens';
import {nestsToken} from './nestsTokens';
import {renderTemplate} from './renderTemplate';
let tempStr = `
  <ul>
    {{#students}}
        <li>
            <dl>
                <dt>{{name}}</dt>
                {{#hobbys}}
                    <dd>{{.}}</dd>
                {{/hobbys}}
            </dl>
        </li>
    {{/students}}
  </ul>
`;
let datas = {
  students: [
    {name: 'Html', hobbys: ['超文本标记语言', '网页结构'], age: 1990, ageThen25: true, show2: true},
    {name: 'Javascript', hobbys: ['弱类型语言', '动态脚本语言', '让页面动起来'], age: 1995, ageThen25: 0, show2: true},
    {name: 'Css', hobbys: ['层叠样式表', '装饰网页', '排版'], age: 1994, ageThen25: 1, show2: true},
  ]
};

let delimiters = [/{{/, /}}/];
var tokens = parseTemplateToTokens(templateStr, delimiters);
console.log(tokens);

var nestedTokens = nestsToken(tokens);
console.log(nestedTokens);

var html = renderTemplate(nestedTokens, datas);
console.log(html);

effect:


7. Existing problems

• I don't know how to implement the function of using operators (such as addition and subtraction, ternary operation) in {{}}
• It is temporarily not supported to current loop item of 1612dda8d11b6a when looping

8. Conclusion

Mustache's token idea is really awesome! ! ! In the future, when we encounter similar requirements, we can also use this idea to achieve it, instead of holding on to regular and string replacement.

Thank you: Thank you Shang Silicon Valley, and Shang Silicon Valley’s Shang Silicon Valley Vue source code analysis series courses, Thank you teacher!