go-zero microservices combat series (1, the beginning)

foreword

In the community, I often see people asking if there is a relatively complete project reference based on go-zero. This type of question is essentially to know the best practices for projects based on go-zero. A complete project should be a complete product function, including product requirements, architecture design, coding of key processes, table design, various performance optimizations and data consistency, etc. It is a project that is really close to production level and can be used directly I refer to it in my own production project, and there is no similar and relatively complete open source project reference in the community at present, so I decided to build a complete project close to production based on go-zero from scratch with you.

The theme of this series is to build a high-concurrency system based on go-zero. Should you choose this theme for Shenma? I guess everyone must feel the boringness of CRUD, but hope that the project is highly concurrent and has certain challenges. Maybe the number of project requests in our work is relatively low, mainly to achieve functions. However, modern Internet products may face the possibility of sudden large traffic at any time. For example, our mall may have relatively stable business traffic at ordinary times, but one day the operation suddenly said that we would do a spike activity to promote it. Concurrent preparation, our service is likely to be unbearable, and the result can be imagined. Therefore, we usually need to reserve enough knowledge of high concurrency in advance, so that we can deal with it calmly when encountering scenarios with high concurrency requirements.

This series is an actual combat series, too basic knowledge may be less involved, so everyone needs to have the following basic abilities:

• To understand the go language syntax and basic usage posture, please refer to the documentation
• The database of this series of projects uses Mysql, please refer to the documentation
• The cache of this series of projects uses Redis, please refer to the documentation
• The message queue of this series of projects uses Kafka, please refer to the documentation
• To understand the basic use of go-zero, please refer to the documentation

Why choose go-zero?

go-zero provides us with many useful tools in high concurrency scenarios. For example, in order to reduce the time consumption of the interface, we often need concurrent request-dependent services. At this time, we can use mapreduce to process requests in parallel; in the face of massive requests, in order to reduce Redis To reduce pressure and improve response time, we can use collection to build a local cache; to avoid service pressure caused by continuous user requests, limit can be used for user-level current limiting and so on. A good framework can bring twice the result with half the effort to my project development. Go-zero's simplicity and ease of use and built-in out-of-the-box tools and service governance capabilities help us build a high-concurrency and high-reliability system.

product demand

In order to be closer to our real project development, the project will be built based on real product requirements. The mall is a system that we usually come into contact with more and more complex, so we decided to build a mall system to learn with you how to build a high-concurrency system based on go-zero. The mall has a lot of function points. We can't demonstrate all the function points one by one, but we will implement an mvp version. In the mvp version , we will put the core functions of go-zero and the core points of building a high-concurrency mall system. Demonstrate it.

The following mind map lists the main functions implemented by the e-commerce system:

Product Prototype

The prototype diagram is the main reference for us to realize the function. The main page UI of the mall system is listed below. Through these UIs, we can roughly construct the system architecture in our mind. Awesome! ! !

Homepage and Category Prototypes

It can be seen that the functions of the home page are relatively complex, including search functions, banner carousel charts, snap-up entries, ranking lists, recommendations, etc. The categories include recommended products of different categories

Cart and My Order prototype

The shopping cart displays a list of items in the shopping cart of the currently logged-in user, including the quantity of items in addition to the basic information of the items. My order list shows a list of items in different states

Product listing and product review prototypes

The product details show the detailed information of the product. At the same time, you can add a shopping cart and buy immediately on this page. Users can also click on the favorite to collect the product. The evaluation function of the product is essential. Are you buying something? When shopping for a product, do you first look at the user reviews of the product?

The main function points implemented by the mall project and the prototype diagram of the main page are posted above. Our project will also be developed around these main function points. We will focus on the core function points, such as how to optimize the homepage banner with a very high request volume, how to ensure that the purchased goods are not oversold, and the realization of distributed transactions during the transaction process.

Series Article Directory

The first article is this article, which mainly gives an overview of the project.

The second article will introduce the division of our microservices, the definition of the project directory structure, and the initialization of some project code using goctl .

The third article will first define the API of the project. Only after defining the API protocol can we really start writing code. How to define the protocol is not clear, which will greatly increase the probability of project rework. After defining the API, we need to design the table structure. , the database we use MySQL.

The fourth article is mainly to familiarize yourself with the basic use of go-zero, including database addition, deletion, modification, and query, cache operations, custom middleware, and the use of common toolkits such as mapreduce.

In the fifth, sixth, and seventh chapters, we focus on the use of caches. The status of caches in high-concurrency systems is self-evident. It is no exaggeration to say that if the cache design is not good, then the high-concurrency systems will no longer exist. We will introduce how to use the cache to optimize the homepage banner with huge requests, implement the product list cache, and how to write the common Cache Aside mode code, as well as cache consistency, breakdown, penetration, avalanche and other optimizations.

The eighth and ninth chapters will introduce how to implement a simple seckill function. The seckill function can be described as a typical representative of high concurrent reading and high concurrent writing. Through these two articles, you can learn the optimization skills and basic posture of the seckill function.

The tenth chapter introduces the problems of distributed transactions that are often encountered in the microservice architecture. After the microservices are split, the microservices monopolize the database and cannot use local transactions. Through this article, you can learn that in go-zero implementation of distributed transactions.

The eleventh article introduces how to write unit tests for our business code.

Chapter 12 Observability of services, services after going online need to be observable, including logs, indicator monitoring, link tracking, etc.

In the thirteenth article, we will deploy the service online and conduct functional testing.

concluding remarks

This article first introduces the background of the series and why we choose to build a high-concurrency system, then introduces the main function points and main page prototype diagrams of the high-concurrency mall system we need to build, and finally lists the A list of article titles in this series, through which you can quickly understand the knowledge system of the entire series.

Building a high-concurrency system is not an easy task, and involves a lot of knowledge points, but I believe that as long as we work together, we will be able to overcome these difficulties and bring our technical capabilities to a higher level. Fortunately, we can stand on the shoulders of giants, and go-zero provides us with a very powerful ability to build high-concurrency services and escort our system.

Hope this article is helpful to you, thank you.

Updated every Monday and Thursday

refer to

https://github.com/zhoushuguang/lebron

project address

https://github.com/zeromicro/go-zero

Welcome go-zero and star support us!

WeChat exchange group

Follow the official account of " Microservice Practice " and click on the exchange group to get the QR code of the community group.