go-zero microservice combat series (5, how to write cache code)

Cache is the foundation of high-concurrency services. It is no exaggeration to say that high-concurrency services cannot be discussed without caching. The cache of this project uses Redis. Redis is the current mainstream cache database and supports rich data types. The underlying collection type mainly depends on five data structures: integer array, doubly linked list, hash table, compressed list and jump table. Redis also provides very powerful performance due to the efficiency of the underlying data structure and the high-performance I/O model based on multiplexing. The following figure shows the underlying data structure corresponding to the Redis data type.

basic use

The function of caching model data is integrated by default in go-zero. When we use goctl to automatically generate model code, we add the -c parameter to generate integrated cached model code.

 goctl model mysql datasource -url="root:123456@tcp(127.0.0.1:3306)/product" -table="*"  -dir="./model" -c

Through simple configuration, we can use the cache of the model layer. The default expiration time of the model layer cache is 7 days. If no data is found, an empty cache will be set. The expiration time of the empty cache is 1 minute. The model layer cache is configured and initialized. as follows:

 CacheRedis:
  - Host: 127.0.0.1:6379
    Type: node

 CategoryModel: model.NewCategoryModel(conn, c.CacheRedis)

The code of this demonstration will be mainly based on the product-rpc service. For simplicity, we use grpcurl to debug directly. Note that the reflection service is mainly registered when starting. The rpc service automatically generated by goctl has been registered for us in the dev or test environment. Now, we need to set our mode to dev, and the default mode to pro, as shown in the following code:

 s := zrpc.MustNewServer(c.RpcServerConf, func(grpcServer *grpc.Server) {
    product.RegisterProductServer(grpcServer, svr)
    if c.Mode == service.DevMode || c.Mode == service.TestMode {
      reflection.Register(grpcServer)
    }
})

Install the grpcurl tool directly using go install, so easy! ! ! Mom don't have to worry anymore that I won't debug gRPC

 go install github.com/fullstorydev/grpcurl/cmd/grpcurl

Start the service, query the service and the method provided by the service through the following commands, you can see that the Product obtains the product details interface and the Products batch obtains the product details interface.

 ~ grpcurl -plaintext 127.0.0.1:8081 list

grpc.health.v1.Health
grpc.reflection.v1alpha.ServerReflection
product.Product

~ grpcurl -plaintext 127.0.0.1:8081 list product.Product
product.Product.Product
product.Product.Products

We first insert some test data into the product table, and the test data is placed in the lebron/sql/data.sql file. At this time, we check the product data with id 1. At this time, there is no data with id 1 in the cache.

 127.0.0.1:6379> EXISTS cache:product:product:id:1
(integer) 0

Use the grpcurl tool to call the Product interface to query the product data with an id of 1, and you can see that the data has been returned

 ~ grpcurl -plaintext -d '{"product_id": 1}' 127.0.0.1:8081 product.Product.Product

{
  "productId": "1",
  "name": "夹克1"
}

Look at the cache of this data with id 1 already exists in redis, this is the cache automatically generated by the framework for us

 127.0.0.1:6379> get cache:product:product:id:1

{\"Id\":1,\"Cateid\":2,\"Name\":\"\xe5\xa4\xb9\xe5\x85\x8b1\",\"Subtitle\":\"\xe5\xa4\xb9\xe5\x85\x8b1\",\"Images\":\"1.jpg,2.jpg,3.jpg\",\"Detail\":\"\xe8\xaf\xa6\xe6\x83\x85\",\"Price\":100,\"Stock\":10,\"Status\":1,\"CreateTime\":\"2022-06-17T17:51:23Z\",\"UpdateTime\":\"2022-06-17T17:51:23Z\"}

We then request the product with id 666, because there is no product with id 666 in our table, the framework will help us cache a null value, the expiration time of this null value is 1 minute

 127.0.0.1:6379> get cache:product:product:id:666
"*"

When we delete data or update data, the row record cache with id as key will be deleted

cache index

Our classified product list needs to support pagination. By sliding up, the next page can be loaded continuously. The products are returned to the list in reverse order of creation time, and the page is paginated by the cursor.

How to store classified items in the cache? We use Sorted Set to store, and member is the id of the product, that is, we only store the cache index in the Sorted Set. After the cache index is found, because we automatically generate a cache with the primary key id index as the key, after finding the index list We can then query the row record cache to get the details of the product. The score of the Sorted Set is the creation time of the product.

Let's analyze how to write the logic of the classified product list. First, read the product id index of the current page from the cache, and call the cacheProductList method. Note that calling the query cache method here ignores the error. Why ignore this error? Because what we expect is to return data to the user as much as possible, that is, if redis hangs up, we will query data from the database and return it to the user, instead of returning an error because redis hangs up.

 pids, _ := l.cacheProductList(l.ctx, in.CategoryId, in.Cursor, int64(in.Ps))

The cacheProductList method is implemented as follows, reads data from the cache in reverse order through ZrevrangebyscoreWithScoresAndLimitCtx, and limits the number of reads to the page size

 func (l *ProductListLogic) cacheProductList(ctx context.Context, cid int32, cursor, ps int64) ([]int64, error) {
  pairs, err := l.svcCtx.BizRedis.ZrevrangebyscoreWithScoresAndLimitCtx(ctx, categoryKey(cid), cursor, 0, 0, int(ps))
  if err != nil {
    return nil, err
  }
  var ids []int64
  for _, pair := range pairs {
    id, _ := strconv.ParseInt(pair.Key, 10, 64)
    ids = append(ids, id)
  }
  return ids, nil
}

In order to indicate the end of the list, we will set an end marker in the Sorted Set, the member of the marker is -1 and the score is 0, so after we find the data from the cache, we need to determine whether the last item of the data is -1, if it is -1, it means that the list has been loaded to the last page. If the user slides the screen again, the front-end will not continue to request the back-end interface. The logic is as follows. After checking the data from the cache, it is queried based on the primary key id. Product details

 pids, _ := l.cacheProductList(l.ctx, in.CategoryId, in.Cursor, int64(in.Ps))
if len(pids) == int(in.Ps) {
  isCache = true
  if pids[len(pids)-1] == -1 {
    isEnd = true
  }
}

If the data found from the cache is 0, then we will query the data under this category from the database. It should be noted here that we need to limit the number of queries when querying data from the database. We default to 300 queries at a time, because We have a page size of 10, 300 entries can let users turn down 30 pages, in most cases users won't turn that many pages at all, so we don't load them all to reduce our cache resources, when users really page more than After 30 pages, we load it into the cache on demand

 func (m *defaultProductModel) CategoryProducts(ctx context.Context, cateid, ctime, limit int64) ([]*Product, error) {
  var products []*Product
  err := m.QueryRowsNoCacheCtx(ctx, &products, fmt.Sprintf("select %s from %s where cateid=? and status=1 and create_time<? order by create_time desc limit ?", productRows, m.table), cateid, ctime, limit)
  if err != nil {
    return nil, err
  }
  return products, nil
}

After getting the data of the current page, we also need to do deduplication, because if we only use createTime as the cursor, it is very likely that the data will be repeated, so we also need to add id as the deduplication condition. The deduplication logic is as follows

 for k, p := range firstPage {
      if p.CreateTime == in.Cursor && p.ProductId == in.ProductId {
        firstPage = firstPage[k:]
        break
      }
}

Finally, if the cache is not hit, we need to write the data found from the database into the cache. It should be noted here that if the data has reached the end, the identifier of the end of the data needs to be added, that is, val is -1, score is 0, Here we write asynchronously to the cache, because the write cache is not the main logic, there is no need to wait for completion, and the write failure has no effect. The asynchronous method reduces the time consumption of the interface, and there are small optimizations everywhere.

 if !isCache {
    threading.GoSafe(func() {
      if len(products) < defaultLimit && len(products) > 0 {
        endTime, _ := time.Parse("2006-01-02 15:04:05", "0000-00-00 00:00:00")
        products = append(products, &model.Product{Id: -1, CreateTime: endTime})
      }
      _ = l.addCacheProductList(context.Background(), products)
    })
}

It can be seen that it is relatively complicated to write a complete logic based on cursor paging. There are many details to consider. You must be careful when writing similar code. The overall code of this method is as follows:

 func (l *ProductListLogic) ProductList(in *product.ProductListRequest) (*product.ProductListResponse, error) {
  _, err := l.svcCtx.CategoryModel.FindOne(l.ctx, int64(in.CategoryId))
  if err == model.ErrNotFound {
    return nil, status.Error(codes.NotFound, "category not found")
  }
  if in.Cursor == 0 {
    in.Cursor = time.Now().Unix()
  }
  if in.Ps == 0 {
    in.Ps = defaultPageSize
  }
  var (
    isCache, isEnd   bool
    lastID, lastTime int64
    firstPage        []*product.ProductItem
    products         []*model.Product
  )
  pids, _ := l.cacheProductList(l.ctx, in.CategoryId, in.Cursor, int64(in.Ps))
  if len(pids) == int(in.Ps) {
    isCache = true
    if pids[len(pids)-1] == -1 {
      isEnd = true
    }
    products, err := l.productsByIds(l.ctx, pids)
    if err != nil {
      return nil, err
    }
    for _, p := range products {
      firstPage = append(firstPage, &product.ProductItem{
        ProductId:  p.Id,
        Name:       p.Name,
        CreateTime: p.CreateTime.Unix(),
      })
    }
  } else {
    var (
      err   error
      ctime = time.Unix(in.Cursor, 0).Format("2006-01-02 15:04:05")
    )
    products, err = l.svcCtx.ProductModel.CategoryProducts(l.ctx, ctime, int64(in.CategoryId), defaultLimit)
    if err != nil {
      return nil, err
    }
    var firstPageProducts []*model.Product
    if len(products) > int(in.Ps) {
      firstPageProducts = products[:int(in.Ps)]
    } else {
      firstPageProducts = products
      isEnd = true
    }
    for _, p := range firstPageProducts {
      firstPage = append(firstPage, &product.ProductItem{
        ProductId:  p.Id,
        Name:       p.Name,
        CreateTime: p.CreateTime.Unix(),
      })
    }
  }
  if len(firstPage) > 0 {
    pageLast := firstPage[len(firstPage)-1]
    lastID = pageLast.ProductId
    lastTime = pageLast.CreateTime
    if lastTime < 0 {
      lastTime = 0
    }
    for k, p := range firstPage {
      if p.CreateTime == in.Cursor && p.ProductId == in.ProductId {
        firstPage = firstPage[k:]
        break
      }
    }
  }
  ret := &product.ProductListResponse{
    IsEnd:     isEnd,
    Timestamp: lastTime,
    ProductId: lastID,
    Products:  firstPage,
  }
  if !isCache {
    threading.GoSafe(func() {
      if len(products) < defaultLimit && len(products) > 0 {
        endTime, _ := time.Parse("2006-01-02 15:04:05", "0000-00-00 00:00:00")
        products = append(products, &model.Product{Id: -1, CreateTime: endTime})
      }
      _ = l.addCacheProductList(context.Background(), products)
    })
  }
  return ret, nil
}

When we request the ProductList interface through the grpcurl tool, the returned data is also written into the cache index. When the next request is made, it will be read directly from the cache.

 grpcurl -plaintext -d '{"category_id": 8}' 127.0.0.1:8081 product.Product.ProductList

cache breakdown

Cache breakdown refers to accessing a very hot data, but the cache does not exist, resulting in a large number of requests being sent to the database, which will lead to a sharp increase in database pressure. Cache breakdown often occurs when hot data expires, as shown in the following figure:

Since cache breakdown often occurs when hotspot data expires, it's better not to let the cache expire. Don't use Exists to determine whether the key exists every time you query the cache, but use Expire to renew the cache. Expire returns the result to determine whether the key exists. Since it is hotspot data, it will not expire through continuous renewal.

Another simple and effective method is to control it through singleflight. The principle of singleflight is that when many requests come at the same time, only one request will finally access the resource, and other requests will wait for the result and then return. An example of obtaining product details and using singleflight for protection is as follows:

 func (l *ProductLogic) Product(in *product.ProductItemRequest) (*product.ProductItem, error) {
  v, err, _ := l.svcCtx.SingleGroup.Do(fmt.Sprintf("product:%d", in.ProductId), func() (interface{}, error) {
    return l.svcCtx.ProductModel.FindOne(l.ctx, in.ProductId)
  })
  if err != nil {
    return nil, err
  }
  p := v.(*model.Product)
  return &product.ProductItem{
    ProductId: p.Id,
    Name:      p.Name,
  }, nil
}

cache penetration

Cache penetration means that the data to be accessed is neither in the cache nor in the database, resulting in a cache miss when the request is accessing the cache, and when accessing the database again, it is found that there is no data to be accessed in the database. At this time, there is no way to read data from the database and write it into the cache to serve subsequent requests. If there are too many similar requests, it will bring huge pressure on the cache and database.

For the cache penetration problem, the solution is actually very simple. It is to cache a null value to avoid transparent transmission to the database every time. The cache time can be set to a shorter time, such as 1 minute. In fact, it has been mentioned above. When we access When the data does not exist, the go-zero framework will automatically add an empty cache for us. For example, when we access the product with the id of 999, the product does not exist in the database.

 grpcurl -plaintext -d '{"product_id": 999}' 127.0.0.1:8081 product.Product.Product

Check the cache at this time, I have already added an empty cache for me

 127.0.0.1:6379> get cache:product:product:id:999
"*"

Cache Avalanche

Cache avalanche means that a large number of application requests cannot be processed in the Redis cache, and then the application sends a large number of requests to the database, causing the database to be suspended, which is so miserable! ! Cache avalanches are generally caused by two reasons, and the solutions are not the same.

The first reason is: a large amount of data in the cache expires at the same time, resulting in a large number of requests that cannot be processed normally.

For the cache avalanche problem caused by the simultaneous invalidation of a large amount of data, the general solution is to avoid setting the same expiration time for a large amount of data. If there is a business requirement for the data to be invalidated at the same time, you can add a smaller expiration time to the expiration time. Random numbers, so that different data have different expiration times, but the difference is not large, so that a large number of data will not expire at the same time, and it can basically meet the needs of the business.

The second reason is: Redis is down and cannot respond to requests normally, which will lead to a large number of requests hitting the database directly, resulting in an avalanche

For these reasons, we generally need to let our database support circuit breaker, so that when the pressure on the database is relatively high, the circuit breaker is triggered, and some requests are discarded. Of course, circuit breaker is detrimental to the business.

The database client of go-zero supports fuse, as follows in the ExecCtx method to use fuse for protection

 func (db *commonSqlConn) ExecCtx(ctx context.Context, q string, args ...interface{}) (
  result sql.Result, err error) {
  ctx, span := startSpan(ctx, "Exec")
  defer func() {
    endSpan(span, err)
  }()

  err = db.brk.DoWithAcceptable(func() error {
    var conn *sql.DB
    conn, err = db.connProv()
    if err != nil {
      db.onError(err)
      return err
    }

    result, err = exec(ctx, conn, q, args...)
    return err
  }, db.acceptable)

  return
}

concluding remarks

This article first introduces the basic posture of cache use in go-zero, then introduces in detail the paging function of the cursor through the cache index, and then introduces the concepts and solutions of cache breakdown, cache penetration, and cache avalanche. Cache is the top priority for high-concurrency systems, but there are still many pits in the use of cache. You must be very careful in normal project development. If you use it improperly, not only will it not bring performance improvement, but it will make business Code gets complicated.

Here I would like to thank @group and @search, two students from the go-zero community, the most beautiful soul, who actively participated in the development of the project and put forward many suggestions for improvement.

Hope this article is helpful to you, thank you.

Updated every Monday and Thursday

Code repository: https://github.com/zhoushuguang/lebron

project address

https://github.com/zeromicro/go-zero

Welcome go-zero and star support us!

WeChat exchange group

Follow the official account of " Microservice Practice " and click on the exchange group to get the QR code of the community group.