Go a library of gorilla/handlers every day

Introduction

In the previous article, we introduced the routing management library gorilla/mux in the gorilla web development kit. At the end of the article, we introduced how to use middleware to handle common logic. In daily Go Web development, developers encounter many of the same middleware requirements. gorilla/handlers (hereinafter referred to as handlers ) collects some of the more commonly used middleware. Let's take a look~

Regarding middleware, the previous articles have already introduced a lot. I won't go into details here. handlers library can be used in the standard library net/http and all frameworks that support the http.Handler interface. Since gorilla/mux also supports the http.Handler interface, it can also be used in conjunction with the handlers library. This is the benefit of the compatible standard .

Project initialization & installation

The code in this article uses Go Modules.

Create a directory and initialize:

$ mkdir gorilla/handlers && cd gorilla/handlers
$ go mod init github.com/darjun/go-daily-lib/gorilla/handlers

Install the gorilla/handlers library:

$ go get -u github.com/gorilla/handlers

The following describes each middleware and the corresponding source code in turn.

Log

handlers provides two log middleware:

• LoggingHandler : an Apache Common Log Format log log HTTP request format;
• CombinedLoggingHandler : an Apache Combined Log Format logging HTTP request log format, Apache and Nginx use this default log format.

The two log formats have very little difference. The Common Log Format is as follows:

%h %l %u %t "%r" %>s %b

The meaning of each indicator is as follows:

• %h : the IP address or host name of the client;
• %l : RFC 1413 defined client identified by the client machine identd generator. If it does not exist, the field is - ;
• %u : The authenticated user name. If it does not exist, the field is - ;

• %t : Time, the format is day/month/year:hour:minute:second zone , where:

  • day : 2 digits;
  • month : month abbreviation, 3 letters, such as Jan ;
  • year : 4 digits;
  • hour : 2 digits;
  • minute : 2 digits;
  • second : 2 digits;
  • zone : + or - followed by 4 digits;
  • For example: 21/Jul/2021:06:27:33 +0800
• %r : Contains HTTP request line information, for example, GET /index.html HTTP/1.1 ;
• %>s : The status code sent by the server to the client, such as 200 ;
• %b : Response length (number of bytes).

Combined Log Format format of 060ff539411831 is as follows:

%h %l %u %t "%r" %>s %b "%{Referer}i" "%{User-Agent}i"

It can be seen that Common Log Format is just more than 060ff539411858:

• %{Referer}i Referer information in the HTTP header;
• %{User-Agent}i User-Agent information in the HTTP header.

For middleware, we can make it act on the whole world, that is, all processors, or make it take effect only on certain processors. If you want to take effect for all processors, you can call the Use() method. If you only need to act on a specific processor, use middleware to wrap the processor in one layer during registration:

func index(w http.ResponseWriter, r *http.Request) {
  fmt.Fprintln(w, "Hello World")
}

type greeting string

func (g greeting) ServeHTTP(w http.ResponseWriter, r *http.Request) {
  fmt.Fprintf(w, "Welcome, %s", g)
}

func main() {
  r := mux.NewRouter()
  r.Handle("/", handlers.LoggingHandler(os.Stdout, http.HandlerFunc(index)))
  r.Handle("/greeting", handlers.CombinedLoggingHandler(os.Stdout, greeting("dj")))

  http.Handle("/", r)
  log.Fatal(http.ListenAndServe(":8080", nil))
}

In the above code, LoggingHandler only acts on the processing function index , and CombinedLoggingHandler only acts on the processor greeting("dj") .

Run the code and visit localhost:8080 and localhost:8080/greeting through a browser:

::1 - - [21/Jul/2021:06:39:45 +0800] "GET / HTTP/1.1" 200 12
::1 - - [21/Jul/2021:06:39:54 +0800] "GET /greeting HTTP/1.1" 200 11 "" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.164 Safari/537.36"

Compared with the indicators analyzed earlier, it is easy to see the various parts.

Since *mux.Router the Use() method accepts type MiddlewareFunc middleware:

type MiddlewareFunc func(http.Handler) http.Handler

But handlers.LoggingHandler/CombinedLoggingHandler is not satisfied, so a layer of packaging is needed to pass it to the Use() method:

func Logging(handler http.Handler) http.Handler {
  return handlers.CombinedLoggingHandler(os.Stdout, handler)
}

func main() {
  r := mux.NewRouter()
  r.Use(Logging)
  r.HandleFunc("/", index)
  r.Handle("/greeting/", greeting("dj"))

  http.Handle("/", r)
  log.Fatal(http.ListenAndServe(":8080", nil))
}

In addition, handlers also provides CustomLoggingHandler , we can use it to define our own log middleware:

func CustomLoggingHandler(out io.Writer, h http.Handler, f LogFormatter) http.Handler

The most critical LogFormatter type definition:

type LogFormatterParams struct {
  Request    *http.Request
  URL        url.URL
  TimeStamp  time.Time
  StatusCode int
  Size       int
}

type LogFormatter func(writer io.Writer, params LogFormatterParams)

We implement a simple LogFormatter , recording time + request line + response code:

func myLogFormatter(writer io.Writer, params handlers.LogFormatterParams) {
  var buf bytes.Buffer
  buf.WriteString(time.Now().Format("2006-01-02 15:04:05 -0700"))
  buf.WriteString(fmt.Sprintf(` "%s %s %s" `, params.Request.Method, params.URL.Path, params.Request.Proto))
  buf.WriteString(strconv.Itoa(params.StatusCode))
  buf.WriteByte('\n')

  writer.Write(buf.Bytes())
}

func Logging(handler http.Handler) http.Handler {
  return handlers.CustomLoggingHandler(os.Stdout, handler, myLogFormatter)
}

use:

func main() {
  r := mux.NewRouter()
  r.Use(Logging)
  r.HandleFunc("/", index)
  r.Handle("/greeting/", greeting("dj"))

  http.Handle("/", r)
  log.Fatal(http.ListenAndServe(":8080", nil))
}

The log currently recorded is in the following format:

2021-07-21 07:03:18 +0800 "GET /greeting/ HTTP/1.1" 200

Look at the source code, we can find LoggingHandler/CombinedLoggingHandler/CustomLoggingHandler are based on the underlying loggingHandler achieve, except that LoggingHandler use predefined writeLog as LogFormatter , CombinedLoggingHandler using predefined writeCombinedLog as LogFormatter , and CustomLoggingHandler use our own definition of LogFormatter :

func CombinedLoggingHandler(out io.Writer, h http.Handler) http.Handler {
  return loggingHandler{out, h, writeCombinedLog}
}

func LoggingHandler(out io.Writer, h http.Handler) http.Handler {
  return loggingHandler{out, h, writeLog}
}

func CustomLoggingHandler(out io.Writer, h http.Handler, f LogFormatter) http.Handler {
  return loggingHandler{out, h, f}
}

The predefined writeLog/writeCombinedLog as follows:

func writeLog(writer io.Writer, params LogFormatterParams) {
  buf := buildCommonLogLine(params.Request, params.URL, params.TimeStamp, params.StatusCode, params.Size)
  buf = append(buf, '\n')
  writer.Write(buf)
}

func writeCombinedLog(writer io.Writer, params LogFormatterParams) {
  buf := buildCommonLogLine(params.Request, params.URL, params.TimeStamp, params.StatusCode, params.Size)
  buf = append(buf, ` "`...)
  buf = appendQuoted(buf, params.Request.Referer())
  buf = append(buf, `" "`...)
  buf = appendQuoted(buf, params.Request.UserAgent())
  buf = append(buf, '"', '\n')
  writer.Write(buf)
}

They are all based on buildCommonLogLine construct basic information. writeCombinedLog also calls http.Request.Referer() and http.Request.UserAgent obtain Referer and User-Agent information.

loggingHandler defined as follows:

type loggingHandler struct {
  writer    io.Writer
  handler   http.Handler
  formatter LogFormatter
}

loggingHandler implementation of 060ff539411b6b is clever: In order to record the response code and response size, a type responseLogger defined to wrap the original http.ResponseWriter , and the information is recorded when writing:

type responseLogger struct {
  w      http.ResponseWriter
  status int
  size   int
}

func (l *responseLogger) Write(b []byte) (int, error) {
  size, err := l.w.Write(b)
  l.size += size
  return size, err
}

func (l *responseLogger) WriteHeader(s int) {
  l.w.WriteHeader(s)
  l.status = s
}

func (l *responseLogger) Status() int {
  return l.status
}

func (l *responseLogger) Size() int {
  return l.size
}

loggingHandler key method of ServeHTTP() :

func (h loggingHandler) ServeHTTP(w http.ResponseWriter, req *http.Request) {
  t := time.Now()
  logger, w := makeLogger(w)
  url := *req.URL

  h.handler.ServeHTTP(w, req)
  if req.MultipartForm != nil {
    req.MultipartForm.RemoveAll()
  }

  params := LogFormatterParams{
    Request:    req,
    URL:        url,
    TimeStamp:  t,
    StatusCode: logger.Status(),
    Size:       logger.Size(),
  }

  h.formatter(h.writer, params)
}

Construct the LogFormatterParams object and call the corresponding LogFormatter function.

compression

If there is a Accept-Encoding header in the client request, the server can use the algorithm indicated by the header to compress the response to save network traffic. handlers.CompressHandler middleware enables the compression function. There is also a CompressHandlerLevel can specify the compression level. In fact, CompressHandler is gzip.DefaultCompression called using CompressHandlerLevel :

func CompressHandler(h http.Handler) http.Handler {
  return CompressHandlerLevel(h, gzip.DefaultCompression)
}

Look at the code:

func index(w http.ResponseWriter, r *http.Request) {
  fmt.Fprintln(w, "Hello World")
}

type greeting string

func (g greeting) ServeHTTP(w http.ResponseWriter, r *http.Request) {
  fmt.Fprintf(w, "Welcome, %s", g)
}

func main() {
  r := mux.NewRouter()
  r.Use(handlers.CompressHandler)
  r.HandleFunc("/", index)
  r.Handle("/greeting/", greeting("dj"))

  http.Handle("/", r)
  log.Fatal(http.ListenAndServe(":8080", nil))
}

Run, request localhost:8080 , and you can see that the response is compressed with gzip through the Network tab of Chrome Developer Tools:

Ignoring some details, the CompressHandlerLevel is as follows:

func CompressHandlerLevel(h http.Handler, level int) http.Handler {
  const (
    gzipEncoding  = "gzip"
    flateEncoding = "deflate"
  )

  return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
    var encoding string
    for _, curEnc := range strings.Split(r.Header.Get(acceptEncoding), ",") {
      curEnc = strings.TrimSpace(curEnc)
      if curEnc == gzipEncoding || curEnc == flateEncoding {
        encoding = curEnc
        break
      }
    }

    if encoding == "" {
      h.ServeHTTP(w, r)
      return
    }

    if r.Header.Get("Upgrade") != "" {
      h.ServeHTTP(w, r)
      return
    }

    var encWriter io.WriteCloser
    if encoding == gzipEncoding {
      encWriter, _ = gzip.NewWriterLevel(w, level)
    } else if encoding == flateEncoding {
      encWriter, _ = flate.NewWriter(w, level)
    }
    defer encWriter.Close()

    w.Header().Set("Content-Encoding", encoding)
    r.Header.Del(acceptEncoding)

    cw := &compressResponseWriter{
      w:          w,
      compressor: encWriter,
    }

    w = httpsnoop.Wrap(w, httpsnoop.Hooks{
      Write: func(httpsnoop.WriteFunc) httpsnoop.WriteFunc {
        return cw.Write
      },
      WriteHeader: func(httpsnoop.WriteHeaderFunc) httpsnoop.WriteHeaderFunc {
        return cw.WriteHeader
      },
      Flush: func(httpsnoop.FlushFunc) httpsnoop.FlushFunc {
        return cw.Flush
      },
      ReadFrom: func(rff httpsnoop.ReadFromFunc) httpsnoop.ReadFromFunc {
        return cw.ReadFrom
      },
    })

    h.ServeHTTP(w, r)
  })
}

Obtain the compression algorithm indicated by the client from the Accept-Encoding If the client does not specify it, or if there is Upgrade request header, it will not be compressed. Otherwise, it is compressed. According to the recognized compression algorithm, create a io.Writer implementation object gzip or flate

Like the previous log middleware, in order to compress the written content, the new type compressResponseWriter encapsulates http.ResponseWriter , the Write() method is rewritten, and the written byte stream is passed to the previously created io.Writer achieve compression:

type compressResponseWriter struct {
  compressor io.Writer
  w          http.ResponseWriter
}

func (cw *compressResponseWriter) Write(b []byte) (int, error) {
  h := cw.w.Header()
  if h.Get("Content-Type") == "" {
    h.Set("Content-Type", http.DetectContentType(b))
  }
  h.Del("Content-Length")

  return cw.compressor.Write(b)
}

Content type

We can use handler.ContentTypeHandler specify that the requested Content-Type must be in the type we give, only valid for the POST/PUT/PATCH method. For example, we restrict the login request to be application/x-www-form-urlencoded in the form of 060ff539411d7a:

func main() {
  r := mux.NewRouter()
  r.HandleFunc("/", index)
  r.Methods("GET").Path("/login").HandlerFunc(login)
  r.Methods("POST").Path("/login").
    Handler(handlers.ContentTypeHandler(http.HandlerFunc(dologin), "application/x-www-form-urlencoded"))

  http.Handle("/", r)
  log.Fatal(http.ListenAndServe(":8080", nil))
}

In this way, as long as the requested /login of Content-Type is not application/x-www-form-urlencoded 415 error will be returned. We can deliberately make a mistake, and then ask to see the performance:

Unsupported content type "application/x-www-form-urlencoded"; expected one of ["application/x-www-from-urlencoded"]

ContentTypeHandler is very simple:

func ContentTypeHandler(h http.Handler, contentTypes ...string) http.Handler {
  return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
    if !(r.Method == "PUT" || r.Method == "POST" || r.Method == "PATCH") {
      h.ServeHTTP(w, r)
      return
    }

    for _, ct := range contentTypes {
      if isContentType(r.Header, ct) {
        h.ServeHTTP(w, r)
        return
      }
    }
    http.Error(w, fmt.Sprintf("Unsupported content type %q; expected one of %q", r.Header.Get("Content-Type"), contentTypes), http.StatusUnsupportedMediaType)
  })
}

It is to read the Content-Type header to determine whether it is in the type we specified.

Method handler

In the above example, the GET and POST methods of the /login r.Methods("GET").Path("/login").HandlerFunc(login) which is a lengthy way of writing. handlers.MethodHandler can simplify this writing:

func main() {
  r := mux.NewRouter()
  r.HandleFunc("/", index)
  r.Handle("/login", handlers.MethodHandler{
    "GET":  http.HandlerFunc(login),
    "POST": http.HandlerFunc(dologin),
  })

  http.Handle("/", r)
  log.Fatal(http.ListenAndServe(":8080", nil))
}

MethodHandler bottom layer of map[string]http.Handler type, and its ServeHTTP() method calls different processing according to the requested Method:

type MethodHandler map[string]http.Handler

func (h MethodHandler) ServeHTTP(w http.ResponseWriter, req *http.Request) {
  if handler, ok := h[req.Method]; ok {
    handler.ServeHTTP(w, req)
  } else {
    allow := []string{}
    for k := range h {
      allow = append(allow, k)
    }
    sort.Strings(allow)
    w.Header().Set("Allow", strings.Join(allow, ", "))
    if req.Method == "OPTIONS" {
      w.WriteHeader(http.StatusOK)
    } else {
      http.Error(w, "Method not allowed", http.StatusMethodNotAllowed)
    }
  }
}

If the method is not registered, it returns 405 Method Not Allowed . Except for one method, OPTIONS . This method returns which methods are supported Allow

Redirect

handlers.CanonicalHost can redirect the request to the specified domain name and specify the redirection response code at the same time. It is useful when the same server corresponds to multiple domain names:

func index(w http.ResponseWriter, r *http.Request) {
  fmt.Fprintln(w, "hello world")
}

func main() {
  r := mux.NewRouter()
  r.Use(handlers.CanonicalHost("http://www.gorillatoolkit.org", 302))
  r.HandleFunc("/", index)
  http.Handle("/", r)
  log.Fatal(http.ListenAndServe(":8080", nil))
}

The above redirects all requests to http://www.gorillatoolkit.org with 302.

CanonicalHost is also very simple:

func CanonicalHost(domain string, code int) func(h http.Handler) http.Handler {
  fn := func(h http.Handler) http.Handler {
    return canonical{h, domain, code}
  }

  return fn
}

Key type canonical :

type canonical struct {
  h      http.Handler
  domain string
  code   int
}

Core method:

func (c canonical) ServeHTTP(w http.ResponseWriter, r *http.Request) {
  dest, err := url.Parse(c.domain)
  if err != nil {
    c.h.ServeHTTP(w, r)
    return
  }

  if dest.Scheme == "" || dest.Host == "" {
    c.h.ServeHTTP(w, r)
    return
  }

  if !strings.EqualFold(cleanHost(r.Host), dest.Host) {
    dest := dest.Scheme + "://" + dest.Host + r.URL.Path
    if r.URL.RawQuery != "" {
      dest += "?" + r.URL.RawQuery
    }
    http.Redirect(w, r, dest, c.code)
    return
  }

  c.h.ServeHTTP(w, r)
}

From the source code, it can be seen that the domain name is illegal or the request that does not specify the protocol (Scheme) or domain name (Host) will not be forwarded.

Recovery

Before, we implemented the PanicRecover middleware ourselves to avoid panic during request processing. handlers provides a RecoveryHandler can be used directly:

func PANIC(w http.ResponseWriter, r *http.Request) {
  panic(errors.New("unexpected error"))
}

func main() {
  r := mux.NewRouter()
  r.Use(handlers.RecoveryHandler(handlers.PrintRecoveryStack(true)))
  r.HandleFunc("/", PANIC)
  http.Handle("/", r)
  log.Fatal(http.ListenAndServe(":8080", nil))
}

Option PrintRecoveryStack means output stack information when panic.

RecoveryHandler is basically the same as the one we wrote before:

type recoveryHandler struct {
  handler    http.Handler
  logger     RecoveryHandlerLogger
  printStack bool
}

func (h recoveryHandler) ServeHTTP(w http.ResponseWriter, req *http.Request) {
  defer func() {
    if err := recover(); err != nil {
      w.WriteHeader(http.StatusInternalServerError)
      h.log(err)
    }
  }()

  h.handler.ServeHTTP(w, req)
}

Summarize

There are many open source Go Web middleware implementations on GitHub, which can be used directly to avoid duplication of wheels. handlers is very lightweight and easy to use in combination net/http and the gorilla routing library mux

If you find a fun and useful Go language library, welcome to submit an issue on the Go Daily Library GitHub😄

refer to

1. gorilla/handlers GitHub：github.com/gorilla/handlers
2. Go daily library GitHub: https://github.com/darjun/go-daily-lib

I

My blog: https://darjun.github.io

Welcome to follow my WeChat public account [GoUpUp], learn together and make progress together~