DevOps Summit | R&D Efficiency Practice Helps Internet Industry Project Management "Effective"

Abstract

This article is the content of Zhang Haoran's speech on "R&D Efficiency Practice Helps Internet Industry Project Management "Effective", which focuses on the two themes of R&D efficiency practice and project management.

When I wrote and shared the PPT, I initially thought about project management for the Internet industry. But now it's not just the Internet, traditional industries are also undergoing digital transformation. Therefore, this project management can be discussed by the whole industry. I used to do research and development, and then I mainly did project management, and I did product management for a period of time in the process. At present, it is mainly in the corporate development department of Netease Youdao, which is responsible for the promotion of the entire R&D efficiency and the improvement of project management.

I will explain my sharing theme in four parts.

1. Common challenges faced by Internet project management

At present, we are in an era of Uka, not only the Internet, but also traditional industries. When any era comes, no one will be abandoned. What are the special - Volatility, uncertainty, complexity and ambiguity.

The so-called volatile means that each iteration should be developed faster, because the market changes rapidly, we are actually not sure where the next direction of the product is and what new demands of users;

The complexity means that there are many forms and structures of products, and it is impossible to use a set of standard solutions to meet the needs of each customer as before. Because customers have also learned to be smart, we need to deal with some more complex and unclear demands. The ambiguity is better understood. Some big companies such as Ali are from e-commerce, and Baidu is a search engine, but the further down the market, the market in each vertical field tends to be saturated, and everyone’s boundaries are getting more and more blurred. Therefore, we must continue to quickly launch our MVP products to gain a head start in the Uka era.

Project management can help enterprises reduce costs and increase efficiency in this era, and quickly verify. However, when we manage the entire software development team, we will encounter some problems or troubles. For example, there is an idea that needs to be done now, and we have formed a team to do R&D iterations quickly. Everyone has to work overtime to catch up. It's hard work, but when the product is actually delivered, there will still be delays.

In the process of research and development, some quality problems will arise in order to catch up with the progress. In the whole process, in order to make up for the problems of delay and poor quality, people will be arrested for effectiveness. During the process of arresting, it is easy to cause some rebuttals from the team, resulting in low morale. If our products are no longer accepted by the market after the launch, The morale of the entire team became even lower over time.

Some people say that I am a very experienced project management or product manager, or technical lead, and I have no problems with the entire scheduling or task assignment process. Then if we have cross-team collaboration, for example, NetEase has a team A, which attaches great importance to the testing process. The testing process is divided into several rounds, and the NetEase EasyTest platform is used to scan and check the code. Another B team, the business form is not so complicated, just simply go through the test process and it will be OK. There are not many automation tools in the process, and they may still use mind maps such as XMIND to write use cases and manage them offline. However, if the partners of team B support the project of team A, they are not familiar with the tools and processes of team A, how can they quickly integrate team B into team A and produce quickly, which will become an obstacle to the efficient output of the team.

Our challenge now is the slow delivery process. Because the more product lines we have, the more products we need to deliver. In different processes, there are some new requirements for a certain product, and many version branches will be established after the requirements come. In this process, for example, when I was developing version A, new requirements were inserted, and if I wanted to open version B, the switch from A to B actually had an impact on development. At the same time, for example, in the early stage of the project, everyone is doing some research and then developing. At this stage, testers are waiting for the development to complete before they can enter the test. The waiting time is too long, which leads to the slow delivery of the whole. Poor quality is also a problem. Before research and development, there is no awareness of testing, and only write your own code and submit it. Some may not test themselves in order to catch up with the progress. The scheduling is unreasonable, and everyone may encounter this problem often. Scheduling is to release the time for research and development, and compress the remaining time into testing time. It only takes a few days, three or four days to test a large part of the functional modules. To catch up with schedule, sacrifice quality.

Collaboration is difficult, with multiple teams lacking standard process tools. In the whole process, one is that it is difficult for personnel to collaborate and switch projects; the other is that if some problems are encountered, there may be strife; finally, there is less measurement. If there is no measurement, it will definitely not work. The project manager, including the management, does not know the current status of the entire team, and the technical leader certainly cannot reasonably assign tasks. Without the measurement of the whole process, there is no way to measure everyone's performance. Now in the implementation of OKR, there is actually a measure, that is, how well the KR is achieved. These frequently asked questions weighed heavily on the team.

In this process, the Matthew effect will be formed. Every time the quality is poor, there is a problem after going online, and the operation and maintenance engineer becomes a "bearer"; the test needs to verify the existing problem as soon as possible, and follow the development to fix it, becoming a "blame man"; finally, our development , can only work overtime and work overtime, and become an "overtime man". This is the "Three Heroes" in the Internet age. The work is very tiring, but the results are not ideal, and there is a lot of noise every day.

2. In view of this situation, how does Youdao do exploration and practice?

first step is to find the problem.

See what's going on at the moment through interviews or observations. Interviews are part of it, and observations must also be included. Because the entire R&D team does not want to be disturbed during the work process, if it disturbs him, firstly, the cooperation may not be enough, and secondly, some answers may not be obtained during the interview process, so the time of the team members is tight. Under the conditions, the problem needs to be found through observation. If you have time, you can invite to do co-creation, or internal co-creation. At the same time, you can share good cases with our team to see their acceptance.

second step is to collect some problems and further analyze and locate the problems.

At that time, we collected problems from requirements to the entire development, construction, testing, deployment, and monitoring.

If there is a demand, there is no demand management tool at the beginning, and everyone relies on written records, which is inefficient, and after the demand is updated, the downstream is unknowable. Failure to do requirement change management will lead to the fact that some of the developed functions are completely different from what they actually want, or are very different, which is a fatal blow to the team. There will be more or less problems in the development, construction, testing and other aspects. We will analyze and locate these problems, including people, processes, and tools.

For these, we have a plan:

First of all, optimize the single-point tools and processes from business to operation and maintenance. The ultimate goal is to establish an end-to-end system that can support the entire R&D process from business requirements to online release, and enable rapid, high-quality and continuous release. s solution.

One is fast, the other is high quality, and the third is continuous delivery. It can't be a quick one, or a good quality one time, to achieve this goal continuously.

first step, the introduction of agile development mode


second step is to create a set of toolchains from the level of the entire enterprise development, and pull the whole situation

third step is to prepare to build your own DevOps energy efficiency platform as an integrated platform.

In the early stage, some process specifications will be passed to make process-based drivers for requirements. We have built a set of standard workflows, in which branch processes are built based on different teams, and try to fall on this set of standard processes. We hope to promote the entire production and research process through some processes, whether it is through cards or through automated tools, or the transformation process, driven by tools, to precipitate this data. At the same time, at the code level and the test level, I did some encapsulation myself, and some did it myself. In terms of CICD, it is not completely self-built, but the front-end encapsulation will be done. Users of our production and research team see that they are using a set of platforms, and the entire process from demand to launch on a set of platforms is more friendly to their perception.
The core process of CI/CD is done with the tekton engine. Through some pipeline combinations, the entire process management is completed, and the entire pod is scheduled through the task file to achieve a continuous pipeline of the entire process.

Based on this level, some constructions have been done from product design to requirements and codes, and sent to the corresponding environment for automated testing, so as to fully realize the connection of the whole link, including the data Kanban after the launch. With this set of full-link tool platforms, we can better support our agile development model, and the last step is stuck in operation and maintenance.

In this case, through this platform, you can get a demand, go online quickly after running, and quickly verify after going online. After verification, whether it is OK or not, it does not meet the release conditions, and the current release is already OK.

3. Effective combination of project management and R&D efficiency

Pull the entire R&D process through and do it through self-construction. How should our project management make some breakthroughs? combined with R&D efficiency. First of all, make some improvements in the way of project management, there is a traditional concept of iron triangle in traditional project management. In terms of project management management, the traditional software industry will pay more attention to the scope, time and cost of the project. For example, I am Party B, and I signed a contract with Party A to deliver some platforms or systems. In this process, there will be strict scope positioning. Our project establishment time will be delivered in several months. The first, second and third phases include our time and cost. When Party A and Party B are connected, they need to do cost control. Project management is more about triggering at these three levels, and then doing process management in the process. But this model actually needs some improvements. Because now is the era of the Internet, the era of Uka, it is impossible for everyone to determine what you want, how long you will be given, and how much budget you will be given at the beginning, so a change needs to be made, which is the concept of the agile triangle.

Scope, time and cost also depends on these three, but the proportion is not so high, but the overall project management as a constraint, that we want a limited scope, time, cost, which sought the most valuable part .

In this process, the quality can be gradually improved through some tools and processes. Through the project management method of the agile triangle, each time delivery is the most valuable, rather than the full range. You can split the smallest unit granularity for mvp based on the value each time. If the value changes, make adjustments quickly.

Second, there are some process improvements to be made in the project management process. From the perspective of project management, it turned out to be a linear process. Generally, after a project is established, it is necessary to do project establishment, feasibility study, feasibility study, design of the entire product, design of UE and UI, research and development, testing, On-line, the whole process is linear. There is a problem, if the product demand PRD is not drawn well, can UE not do it? Is it not possible to develop?

So we improved the process and reduced its lead time. PRD is not fully written, but in this form of user story, first give the whole picture of the whole user story. According to what to do in the end, take out the most valuable part, and then UE and development can meet the requirements of this part first. After the most valuable part is done, the second most valuable function in the prioritization can be implemented. It is equivalent to dismantling the entire function, and subsequent processes can be directly accessed.

In addition, according to the project management plan, there must be a measurement system for management, otherwise it is impossible to know the efficiency point of the entire R&D team, or the situation of work saturation. Everyone is allowed to have a certain amount of "fishing", but it must be done in moderation. When some "fishing" is obviously unreasonable, it needs to be measured.

Finally, in project management, we also gradually find healthy measurement methods and systems, and gradually improve the process.

Short-term measurement is the first step to be done, because there is no link platform statistics for the whole process, and they are all process-based measurements, and only single-point collection of data for each process of demand, development, testing, and operation and maintenance. The project manager is more tired and needs to manually go to various platforms to pick up through the excel sheet, or go to some other places to ask questions, which is more difficult. But with this measurement system, it can basically be driven by this. Which data is more important for everyone to see now, or the collection process is more painful, you can gradually make some improvements.

With a single point of data, the second step is to focus on the delivery target. For example, if you need to deliver 5 or 10 stories this time, this process is based on the overall goal. In the process, the efficiency of some personnel, such as the cycle, the efficiency per person, including the repair time of defects, will be passed through the entire iteration. Look, not look through a need. In addition, for iterative releases, look at the release frequency or release lead time. There may be some links that are not ready before the release. Because of some situations, everyone is anxious to develop at first, and when it is about to go online, apply for testing, or wait for a machine configuration or something, which hinders time. The release time will be very long, resulting in a long time for the final version of the entire iteration to go online. Quality will be done through some smoke rejection rate, defect density and defect missed detection rate and other indicators throughout the process and iteration.

The third step is value-based measurement. There is a goal, but this goal is only the process of realizing the value of the product, and ultimately the value must be realized. Value-oriented, filter the entire link layer by layer, do it at a single point or do it through iteration, dismantle layer by layer, and gradually eliminate it. Guided by the delivery of value, gradually decompose and analyze through visual views or dashboards to improve the efficiency and quality of the entire R&D performance.

Such a combination path will be based on the overall balance of product users and the market, and output the most valuable points this time, which is equivalent to having a set of its own value evaluation system, which will be integrated into the entire production and research system. In the process of production and research, some requirements planning, version management, and time boxes will be done, and the project manager can control the process more effectively.

Our goal is to quickly explore and analyze the business through a complete set of energy efficiency platforms, form an MVP, and quickly realize the launch and verification of the MVP.

4. Future Prospects of R&D Efficiency

1. DevOps continues to deepen, focusing on integrating rich data sources.

It is necessary to pay attention to whether the data is effective, reasonable, and rich enough. It is necessary to integrate data sources in this area, and increase monitoring indicators through analysis. This depth needs to be more reasonable, to promote the efficient work of the R&D team, and not to destroy the team atmosphere.

2. It has been launched now, but a large-scale AIOps has not yet been formed.

Although our company also has AI capabilities and AI teams, AI needs to be done based on big data. Only when data is abundant and reasonableness is high, AIOps will be reasonable and effective. Because only the correct and reasonable data is analyzed, the detected problems and automatic fixes are only reasonable and meaningful. Using unreasonable data to do AI is going in the wrong direction.

Third, the ultimate goal is to do NoOps, which is equivalent to releasing the pressure on the entire production and research side, and fully realize the automation of the assembly line.

That's it for my speech today, thank you all!