Go a library of fasttemplate every day

Introduction

fasttemplate is a relatively simple and easy-to-use small template library. fasttemplate author valyala also a lot of good open source libraries, such as the famous fasthttp , described earlier bytebufferpool , there is a heavyweight template library quicktemplate . quicktemplate much more flexible and easier to use text/template and html/template in the standard library. We will introduce it later. fasttemlate to be introduced today only focuses on a small area-string replacement. Its goal is to replace strings.Replace , fmt.Sprintf and other methods, providing a simple, easy-to-use, high-performance string replacement method.

This article first introduces fasttemplate , and then looks at some details of the source code implementation.

Quick to use

The code in this article uses Go Modules.

Create a directory and initialize:

$ mkdir fasttemplate && cd fasttemplate
$ go mod init github.com/darjun/go-daily-lib/fasttemplate

Install the fasttemplate library:

$ go get -u github.com/valyala/fasttemplate

Write code:

package main

import (
  "fmt"

  "github.com/valyala/fasttemplate"
)

func main() {
  template := `name: {{name}}
age: {{age}}`
  t := fasttemplate.New(template, "{{", "}}")
  s1 := t.ExecuteString(map[string]interface{}{
    "name": "dj",
    "age":  "18",
  })
  s2 := t.ExecuteString(map[string]interface{}{
    "name": "hjw",
    "age":  "20",
  })
  fmt.Println(s1)
  fmt.Println(s2)
}

• Define the template string, use {{ and }} represent placeholders, which can be specified when creating the template;
• Call fasttemplate.New() create a template object t , and pass in the start and end placeholders;
• Call the t.ExecuteString() method of the template object and pass in the parameters. The parameter has the value corresponding to each placeholder. Generate the final string.

operation result:

name: dj
age: 18

We can customize the placeholders, using {{ and }} as the start and end placeholders respectively. We can change to [[ and ]] , just simply modify the code:

template := `name: [[name]]
age: [[age]]`
t := fasttemplate.New(template, "[[", "]]")

Also, note that the type of arguments passed to map[string]interface{} , but fasttemplate accept only type []byte , string and TagFunc value types. This is why the above 18 enclosed in double quotes.

Another point to note is that fasttemplate.New() returns a template object. If the template parsing fails, it will directly panic . If you want to handle the error yourself, you can call the fasttemplate.NewTemplate() method, which returns a template object and an error. In fact, fasttemplate.New() interior is calling fasttemplate.NewTemplate() , if it returns an error, it panic :

// src/github.com/valyala/fasttemplate/template.go
func New(template, startTag, endTag string) *Template {
  t, err := NewTemplate(template, startTag, endTag)
  if err != nil {
    panic(err)
  }
  return t
}

func NewTemplate(template, startTag, endTag string) (*Template, error) {
  var t Template
  err := t.Reset(template, startTag, endTag)
  if err != nil {
    return nil, err
  }
  return &t, nil
}

This is actually an idiom. For example programs that do not want to deal with errors, direct panic sometimes an option . For example, the html.template standard library also provides the Must() method, which is generally used in this way. If the parsing fails, panic :

t := template.Must(template.New("name").Parse("html"))

Do not add spaces in the middle of the ! !

Do not add spaces in the middle of the ! !

Do not add spaces in the middle of the ! !

A shortcut

By using fasttemplate.New() define template objects, we can use different parameters to replace them multiple times. However, sometimes we have to do a large number of one-time replacements, and it is more cumbersome to define the template object each time. fasttemplate also provides a one-time replacement method:

func main() {
  template := `name: [name]
age: [age]`
  s := fasttemplate.ExecuteString(template, "[", "]", map[string]interface{}{
    "name": "dj",
    "age":  "18",
  })
  fmt.Println(s)
}

In this way, we need to pass in the template string, start placeholder, end placeholder and replacement parameters at the same time.

TagFunc

fasttemplate provides a TagFunc , which can add some logic to the replacement. TagFunc is a function:

type TagFunc func(w io.Writer, tag string) (int, error)

When performing the replacement, fasttemplate TagFunc function once for each placeholder, and tag is the name of the placeholder. Look at the following program:

func main() {
  template := `name: {{name}}
age: {{age}}`
  t := fasttemplate.New(template, "{{", "}}")
  s := t.ExecuteFuncString(func(w io.Writer, tag string) (int, error) {
    switch tag {
    case "name":
      return w.Write([]byte("dj"))
    case "age":
      return w.Write([]byte("18"))
    default:
      return 0, nil
    }
  })

  fmt.Println(s)
}

This is actually TagFunc get-started sample program, and different values are written according to the incoming tag If we look at the source code, we will find that in fact ExecuteString() will eventually call ExecuteFuncString() . fasttemplate provides a standard TagFunc :

func (t *Template) ExecuteString(m map[string]interface{}) string {
  return t.ExecuteFuncString(func(w io.Writer, tag string) (int, error) { return stdTagFunc(w, tag, m) })
}

func stdTagFunc(w io.Writer, tag string, m map[string]interface{}) (int, error) {
  v := m[tag]
  if v == nil {
    return 0, nil
  }
  switch value := v.(type) {
  case []byte:
    return w.Write(value)
  case string:
    return w.Write([]byte(value))
  case TagFunc:
    return value(w, tag)
  default:
    panic(fmt.Sprintf("tag=%q contains unexpected value type=%#v. Expected []byte, string or TagFunc", tag, v))
  }
}

The standard TagFunc implementation is also very simple, that is map[string]interface{} and do the corresponding processing. If it is of []byte and string , directly call the write method of io.Writer If it is of TagFunc , call this method directly and pass in io.Writer tag Other types directly throw an error panic

If the tag in the template does not exist in the parameter map[string]interface{} , there are two processing methods:

• Ignore it directly, which is equivalent to replacing it with an empty string "" . The standard stdTagFunc is handled in this way;
• Keep the original tag . keepUnknownTagFunc does this.

keepUnknownTagFunc code of 060b047a731146 is as follows:

func keepUnknownTagFunc(w io.Writer, startTag, endTag, tag string, m map[string]interface{}) (int, error) {
  v, ok := m[tag]
  if !ok {
    if _, err := w.Write(unsafeString2Bytes(startTag)); err != nil {
      return 0, err
    }
    if _, err := w.Write(unsafeString2Bytes(tag)); err != nil {
      return 0, err
    }
    if _, err := w.Write(unsafeString2Bytes(endTag)); err != nil {
      return 0, err
    }
    return len(startTag) + len(tag) + len(endTag), nil
  }
  if v == nil {
    return 0, nil
  }
  switch value := v.(type) {
  case []byte:
    return w.Write(value)
  case string:
    return w.Write([]byte(value))
  case TagFunc:
    return value(w, tag)
  default:
    panic(fmt.Sprintf("tag=%q contains unexpected value type=%#v. Expected []byte, string or TagFunc", tag, v))
  }
}

The second half of the processing is the stdTagFunc , if the first half of the function is not found, tag Write startTag + tag + endTag as the replacement value.

ExecuteString() method we called earlier stdTagFunc , that is, directly tag with an empty string. If you want to keep the unrecognized tag , just call the ExecuteStringStd() method instead. When this method encounters unrecognized tag will remain:

func main() {
  template := `name: {{name}}
age: {{age}}`
  t := fasttemplate.New(template, "{{", "}}")
  m := map[string]interface{}{"name": "dj"}
  s1 := t.ExecuteString(m)
  fmt.Println(s1)

  s2 := t.ExecuteStringStd(m)
  fmt.Println(s2)
}

age is missing in the parameter, the operation result:

name: dj
age:
name: dj
age: {{age}}

Method with io.Writer

The methods described above all return a string at the end. The method name has String : ExecuteString()/ExecuteFuncString() .

We can directly pass in a io.Writer parameter, and call the Write() method of this parameter to write the result string directly. There is no such method name String : Execute()/ExecuteFunc() :

func main() {
  template := `name: {{name}}
age: {{age}}`
  t := fasttemplate.New(template, "{{", "}}")
  t.Execute(os.Stdout, map[string]interface{}{
    "name": "dj",
    "age":  "18",
  })

  fmt.Println()

  t.ExecuteFunc(os.Stdout, func(w io.Writer, tag string) (int, error) {
    switch tag {
    case "name":
      return w.Write([]byte("hjw"))
    case "age":
      return w.Write([]byte("20"))
    }

    return 0, nil
  })
}

Since os.Stdout implements the io.Writer interface, it can be passed in directly. The result is written directly to os.Stdout . run:

name: dj
age: 18
name: hjw
age: 20

Source code analysis

First look at the structure and creation of template objects:

// src/github.com/valyala/fasttemplate/template.go
type Template struct {
  template string
  startTag string
  endTag   string

  texts          [][]byte
  tags           []string
  byteBufferPool bytebufferpool.Pool
}

func NewTemplate(template, startTag, endTag string) (*Template, error) {
  var t Template
  err := t.Reset(template, startTag, endTag)
  if err != nil {
    return nil, err
  }
  return &t, nil
}

After the template is created, the Reset() method will be called to initialize:

func (t *Template) Reset(template, startTag, endTag string) error {
  t.template = template
  t.startTag = startTag
  t.endTag = endTag
  t.texts = t.texts[:0]
  t.tags = t.tags[:0]

  if len(startTag) == 0 {
    panic("startTag cannot be empty")
  }
  if len(endTag) == 0 {
    panic("endTag cannot be empty")
  }

  s := unsafeString2Bytes(template)
  a := unsafeString2Bytes(startTag)
  b := unsafeString2Bytes(endTag)

  tagsCount := bytes.Count(s, a)
  if tagsCount == 0 {
    return nil
  }

  if tagsCount+1 > cap(t.texts) {
    t.texts = make([][]byte, 0, tagsCount+1)
  }
  if tagsCount > cap(t.tags) {
    t.tags = make([]string, 0, tagsCount)
  }

  for {
    n := bytes.Index(s, a)
    if n < 0 {
      t.texts = append(t.texts, s)
      break
    }
    t.texts = append(t.texts, s[:n])

    s = s[n+len(a):]
    n = bytes.Index(s, b)
    if n < 0 {
      return fmt.Errorf("Cannot find end tag=%q in the template=%q starting from %q", endTag, template, s)
    }

    t.tags = append(t.tags, unsafeBytes2String(s[:n]))
    s = s[n+len(b):]
  }

  return nil
}

The initialization does the following things:

• Record start and end placeholders;
• Analyze the template, cut the text and tag separately, and store them in the texts and tags slices respectively. The second half of the for loop is what it does.

Code details:

• First, count the total number of placeholders, and construct the corresponding size text and tag slice at a time. Note that the correctly constructed template string text slice must be tag slice. Like this | text | tag | text | ... | tag | text | ;
• In order to avoid memory copy, use unsafeString2Bytes make the returned byte slice directly point to the internal address of string

Looking at the above introduction, there seem to be many ways. In fact, the core method is ExecuteFunc() . Other methods call it directly or indirectly:

// src/github.com/valyala/fasttemplate/template.go
func (t *Template) Execute(w io.Writer, m map[string]interface{}) (int64, error) {
  return t.ExecuteFunc(w, func(w io.Writer, tag string) (int, error) { return stdTagFunc(w, tag, m) })
}

func (t *Template) ExecuteStd(w io.Writer, m map[string]interface{}) (int64, error) {
  return t.ExecuteFunc(w, func(w io.Writer, tag string) (int, error) { return keepUnknownTagFunc(w, t.startTag, t.endTag, tag, m) })
}

func (t *Template) ExecuteFuncString(f TagFunc) string {
  s, err := t.ExecuteFuncStringWithErr(f)
  if err != nil {
    panic(fmt.Sprintf("unexpected error: %s", err))
  }
  return s
}

func (t *Template) ExecuteFuncStringWithErr(f TagFunc) (string, error) {
  bb := t.byteBufferPool.Get()
  if _, err := t.ExecuteFunc(bb, f); err != nil {
    bb.Reset()
    t.byteBufferPool.Put(bb)
    return "", err
  }
  s := string(bb.Bytes())
  bb.Reset()
  t.byteBufferPool.Put(bb)
  return s, nil
}

func (t *Template) ExecuteString(m map[string]interface{}) string {
  return t.ExecuteFuncString(func(w io.Writer, tag string) (int, error) { return stdTagFunc(w, tag, m) })
}

func (t *Template) ExecuteStringStd(m map[string]interface{}) string {
  return t.ExecuteFuncString(func(w io.Writer, tag string) (int, error) { return keepUnknownTagFunc(w, t.startTag, t.endTag, tag, m) })
}

Execute() method constructs a TagFunc call ExecuteFunc() , which uses stdTagFunc internally:

func(w io.Writer, tag string) (int, error) {
  return stdTagFunc(w, tag, m)
}

ExecuteStd() method constructs a TagFunc call ExecuteFunc() , which uses keepUnknownTagFunc internally:

func(w io.Writer, tag string) (int, error) {
  return keepUnknownTagFunc(w, t.startTag, t.endTag, tag, m)
}

ExecuteString() and ExecuteStringStd() methods call the ExecuteFuncString() method, and the ExecuteFuncString() method calls the ExecuteFuncStringWithErr() method. The ExecuteFuncStringWithErr() method internally uses the bytebufferpool.Get() obtain a bytebufferpoo.Buffer object to call the ExecuteFunc() method. So the core is the ExecuteFunc() method:

func (t *Template) ExecuteFunc(w io.Writer, f TagFunc) (int64, error) {
  var nn int64

  n := len(t.texts) - 1
  if n == -1 {
    ni, err := w.Write(unsafeString2Bytes(t.template))
    return int64(ni), err
  }

  for i := 0; i < n; i++ {
    ni, err := w.Write(t.texts[i])
    nn += int64(ni)
    if err != nil {
      return nn, err
    }

    ni, err = f(w, t.tags[i])
    nn += int64(ni)
    if err != nil {
      return nn, err
    }
  }
  ni, err := w.Write(t.texts[n])
  nn += int64(ni)
  return nn, err
}

The whole logic is very clear, for cycle is Write a texts elements to the current tag execution TagFunc , index +1. Finally, write the last texts element to complete. It looks like this:

| text | tag | text | tag | text | ... | tag | text |

Note: The ExecuteFuncStringWithErr() bytebufferpool introduced in the previous article. If you are interested, you can go back and read it.

to sum up

You can use fasttemplate complete the tasks of strings.Replace and fmt.Sprintf fasttemplate more flexible. The code is clear and easy to understand, and it's worth a look.

Tucao: on naming, Execute() methods which use stdTagFunc , ExecuteStd() which method to use keepUnknownTagFunc method. I think it’s better to stdTagFunc to defaultTagFunc ?

If you find a fun and useful Go language library, welcome to submit an issue on the Go Daily Library GitHub😄

reference

1. fasttemplate GitHub：github.com/valyala/fasttemplate
2. Go daily library GitHub: https://github.com/darjun/go-daily-lib

I

My blog: https://darjun.github.io

Welcome to follow my WeChat public account [GoUpUp], learn together and make progress together~