Where is the Proto code?

Although the company has been switching from a large monomer to a micro-service for a certain number of years, there will always be different people’s opinions on the details of the processing. One of the discussion points is that the IDL code of Proto is placed where?

At present, there are a total of the following solutions, let's discuss the Proto storage method and the corresponding advantages and disadvantages.

Option 1: Store in the code warehouse

Directly store all the Proto files that the project depends on in the proto/ directory, without being automatically pulled and released by development tools:

shortcoming

1. All dependent Protos of the project are stored in the code warehouse, so all dependent Protos need to be manually "requested" from other business groups and placed in the proto/ directory. Manual intervention is extremely troublesome.
2. Proto upgrades and changes often have to repeat the first step, and communication costs are high.

advantage

1. All dependent Protos of the project are stored under the code warehouse, so there is no issue of individual warehouse permissions.
2. The switching overhead of multiple Protos is reduced, because they are all under the code repository, so you don't need to look here and there.

Option 2: Independent warehouse

Independent warehouse storage is our earliest method, that is, each service corresponds to a Proto warehouse:

The advantage of this solution is that all Proto warehouses can be managed independently, and the permissions are clearly divided. But the biggest advantage is also the biggest disadvantage, because a service depends on multiple Proto repositories, and there are cases where it is called across business groups:

As shown in the figure above, the svc-user service relies on three Proto warehouses respectively, which are six Proto warehouses in their own group, business group A, and business group B.

shortcoming

1. Suppose you are a newly hired developer, then you need to find different business groups to apply for different warehouse permissions, which is very troublesome. If there is no batch empowerment tool and no administrator authority, then one needs to be empowered, which is very troublesome.
2. When running the service, you need to pull down all the associated Proto warehouses. If there is no tool for semi-automatic support, the trouble will be unbearable.

advantage

1. Makes the security higher (but IDL itself does not have too many secrets).
2. Pull on demand, no need to pay attention to the rest of the service Proto.

Option 3: Centralized warehouse

Centralized warehouse is also one of the ways that some companies consider. It is to store Proto codes according to the dimensions of the company or large business unit. In this way, you only need to pull a warehouse to obtain all the required IDL:

shortcoming

1. Security is reduced because the IDLs of other business groups are also "leaked".
2. Non-on-demand pull, when viewing the original file, you need to pay attention to some extras.

advantage

1. You only need to pull the Proto warehouse once to easily collect the IDL required for a service.
2. The complexity of warehouse authority management is reduced.

Solution Four: Mirror Warehouse

Combining the characteristics of the above several solutions, we can also get the mirror warehouse method, that is, the Proto file of our service is stored in the proto file of the code warehouse, and after the feature is submitted or released, it is automatically synchronized to the mirror warehouse.

And the other service Proto you depend on is directly obtained by reading the centralized mirror warehouse:

In this way, through the cooperation of development tools, developers only need to pay attention to the Proto of their own project during development, and the centralized mirror warehouse can be used for construction and deployment, which greatly reduces the attention and switching overhead of multiple Protos.

Option Five: Other

In essence, all the above solutions have some pros and cons, and all need development tools to support, otherwise it will be very troublesome to operate.

If you want to get it all done once and for all, you can solve it through the cloud development environment, because when you allocate the cloud development machine, you already have identity authentication, what permissions you can and cannot have, are basically clear, and generally in the group, When cross-group joint debugging, it can also be directly scheduled without paying too much attention like other solutions, and even running a set of microservices locally.

But this requires a lot of tool/resource support.

summary

In this article, I introduced 5 common management methods of Proto code. Each has its advantages and disadvantages. Different companies and different people have different understanding or adaptability. You can choose according to the actual environment, and it is recommended to add the core ones. The personnel discuss and select, because the Proto code is quite extensive, and some people have different opinions.