Brush KPI or real contribution? How to objectively evaluate Huawei's contribution to the Linux kernel?

Recently, an email named "Please don't waste maintainers' time on your KPI grabbing patches (AKA, don't be a KPI jerk)" appeared on the Linux mailing list. Linux kernel maintainer Qu Wenruo pointed out in the email that the Huawei developer submitted a patch that is suspected of brushing KPI , which has caused heated discussions in the community. For details, see: " Linux kernel maintainer approves Huawei developer to brush KPI "

Later, Huawei Linux kernel contributor Leizhen responded to the matter on the mailing list. He mentioned in the email that his main contribution to the kernel in the past was to optimize the performance of the Arm64 SMMU driver, including iova optimization, strict mode optimization, and lazy mode optimization. In addition, he also participated in the development of the Arm SoC driver. When time and energy allowed, he also contributed to other modules of the Linux kernel, trying to find areas for improvement, during which he did some "clean-up" work. Finally, Leizhen said that in the future, he will continue to make more and more important contributions to the Linux community.

Qu Wenruo, the publisher of the original email, quickly replied to Leizhen. He recognized Leizhen’s important contributions to the Linux kernel in the past, and at the same time expressed his attitude towards the “cleanup” work-not unimportant, but please merge these small fixes into a larger patch before submitting. After all, the maintainer's review work is very busy, don't let them waste their time on these insignificant issues. Finally, Qu Wenruo listed some outstanding contributions that are meaningful to the Linux kernel community.

According to the Linux kernel 5.10 development statistics released in December 2020, Huawei ranked first in the number of patches submitted to Linux Kernel 5.10, and the number of modified code lines ranked second, second only to Intel .

So is Huawei’s contribution to the Linux kernel a KPI or is it real? How to objectively evaluate Huawei's contribution to the Linux kernel?

How to objectively evaluate Huawei's contribution to the Linux kernel?

If submitted in accordance with the number of times to count the contribution of , Huawei's contribution second only to the individual developer (gmail.com and kernel.org mail suffix), ranked first in global technology companies.

If we want to block the "KPI brushing" in the email, there are two other ranking methods- according to development equivalent (code logic complexity index) or influence ranking , and new results will be obtained .

If we calculate according to the development equivalent ELOC, , and shield the noise such as blank lines and dead codes , then Huawei's ranking will fall to ten , as shown in the following table. The top three technology companies are Intel, AMD and NVIDIA.

If we dependencies between code be considered to come, the overall influence Sort as follows:

We can see that Huawei is still ranked about ten, which is similar to companies such as Google and Microsoft .

From the shallow indicators such as the number of submissions to the in-depth indicators such as ELOC or Impact, Huawei’s contribution rankings are different. It can be seen that program analysis can provide fresh perspectives and information for the measurement of contributions. evaluates a developer in the project You can’t just look at the number of submissions for your contribution. There are also indicators such as ELOC and Impact , which can better reflect the developer’s contribution to the project and reduce the impact of different development habits on measurement accuracy.

Of course, even according to more scientific evaluation indicators, Huawei still ranks among the top ten global technology companies, and is the domestic company that has made the most contributions. From this perspective, although we encourage core contributions, we can’t think that Huawei’s “first” is the obtained from KPIs across the board.

Finally, we hope to tell beginners or friends who have just participated in open source "Don't do something small without doing good things". Beginners don't need to care too much about whether the contribution is core enough, start contributing more than everything, even if it is just a few typos, it is worthy of praise.

Related terms explained

About ELOC (Development Equivalent)

Development equivalent is a reasonable quantification and measurement of programmer code output. Compared with the shallow statistical indicators such as the number of lines of code and the number of submissions, the advantages of development equivalent are reflected in two aspects: First, it is not easy to be interfered by programming habits or specific code behaviors (such as line breaks, blank lines, comments, brackets, etc.). And can better reflect the amount of logic involved in code development.
Specifically, the development equivalent calculates the complexity of the abstract syntax tree. We can calculate the absolute value of the development equivalent or the cumulative value of the development equivalent. Software development is a dynamic process. As the code changes with submission, the corresponding abstract syntax tree will also evolve.
The absolute value of the development equivalent can be understood as calculating the abstract syntax tree of the code on a submitted aspect, taking into account the height of the abstract syntax tree, the number of nodes, and the weight of different nodes. The absolute value of development equivalent fluctuates up and down with the development process, usually presenting a pattern of "continuous increase-small decrease" and repeated continuously.
The cumulative value of the development equivalent is the calculation of the changes in the code before and after each submission, based on the accumulation of the minimum edit distance between the abstract syntax trees before and after each submission. The code deletion is also considered as a contribution, but the weight will be significantly lower. In the code increase. The cumulative value of development equivalent is a monotonically increasing variable, mainly used to reflect the output and progress of the team or project.

About Impact (Development Value)

Development value is a comprehensive index that integrates the relationship between development equivalent and code call. For ease of understanding, we calculate the index in the form of a percentage, which can be intuitively understood as the contribution ratio.
The calling relationship reflects the interdependent relationship between codes, including function calling, class inheritance, interface calling, etc. The code is not expressed in a nonlinear manner, but is based on a graph composed of dependencies. The more code directly or indirectly depends on a certain piece of code, the higher the influence of the code; it also means that if the piece of code is modified, it will return Correspondingly the larger the scope of the test; generally speaking, the higher the cost and importance of such code modification.

---

Related reading:

• To view the full list, please visit https://ranking.merico.build/oss-orgs/ , the data and analysis tools behind it come from this project https://github.com/merico-dev/build
• Linux kernel maintainer approves Huawei developers to brush KPI
• Knowing: Huawei Linux kernel contributors are questioned about KPI. What is the real situation? What information is worth paying attention to?