Alibaba DevOps Practice Guide | Why is the inevitable trend of DevOps BizDevOps?

Introduction to Starting from lean thinking, we can see the inevitable development direction of DevOps, which is to extend to the business side. Business is the source of product development and operation and maintenance, and a complete value stream must start from the source. This is not a prediction, but a fact that is happening

Editor's note: This article is derived from the "Alibaba DevOps Practice Guide" produced by the Alibaba Cloud Cloud Effects Team. Scan the QR code above or go to: https://developer.aliyun.com/topic/devops , download the full version of the e-book, Learn about Ali's ten-year DevOps practical experience.

Author of this article: He Mian, Senior Technical Expert of Alibaba Cloud Cloud

When it comes to DevOps, we must start with agile and lean development. DevOps developed on their basis, borrowed from the methods and concepts, and developed and improved their practice system.

The rise of agile software development

The practice of agile software development first appeared in the 1990s. At that time, a number of lightweight software engineering methods and frameworks were born one after another. Their common feature is that, compared with traditional software engineering, they all follow the evolution and iterative model, and the process is more lightweight and flexible. Among them, Scrum and Extreme Programming are the most successful in practice and have the greatest impact. They are both iterative and incremental software development frameworks. The difference is that Scrum only includes management practices, while Extreme Programming covers both engineering and management practices.

In the 1990s, the popularity of PC software and the emergence of the fourth generation of programming languages, as well as the rise of object-oriented and design pattern movements, allowed the development of small projects to flourish. At the same time, Internet applications and open source communities have emerged, and development models that are different from traditional ones have continued to emerge. The role of outstanding individuals in program development has been highlighted.

These factors have allowed non-traditional development methods to have experimental soil. As a result, on the one hand, there are endless quality problems, which partly promoted the prosperity and promotion of CMM/CMMI from the total quality management system at this time; on the other hand, it also produced many effective practices that are different from traditional methods. Seeing new possibilities. At this time, the agile movement is ready to emerge. It is not only a rebellion against tradition, but also a norm for barbaric growth.

In February 2001, 17 representatives of lightweight software engineering methods gathered in the Snowbird ski resort in Utah, USA, including several inventors of Scrum and Extreme Programming. http://agilemanifesto.org/) , which had a huge impact, was released after the two-day meeting. The agile manifesto stated their consensus on software development methods, which is equally important They found the word agile to generalize these ideas.

The concept of agile appeared in 2001, and it can be said that it is the right time. At that time, on the one hand, traditional methods became more and more cumbersome, but did not solve the software crisis; on the other hand, human beings are entering the Internet era, and the software industry’s requirements for responding to changes and innovation are rapidly escalating. This is the more fundamental reason. Demand is the best booster for the development of the industry.

After the agile manifesto was released, agile became a movement and was quickly promoted and applied. However, the early agile focus is on the R&D delivery stage. From a business perspective, its goal is to help product and R&D teams improve their agile responsiveness, that is, "deliver value earlier and respond to changes more flexibly". However, in the context of enterprise digital transformation, IT must not only guarantee product development and delivery, but also system deployment and operation. DevOps inherits the concept of agile development and adds the part of operation and maintenance, but DevOps is by no means a simple superposition of development and operation and maintenance, which we will talk about later.

The emergence of lean product development

Lean thinking originated in the field of manufacturing, rooted in Toyota's management and engineering practices in product manufacturing. In 1988, a paper "The Triumph of Lean Production System" in "Sloan Management Review" compared the huge gap in efficiency and quality between Western production methods and Toyota production methods, challenging the myth that large-scale production brings benefits. From then on, lean began to enter the western vision and gradually became an important part of modern management.

The book "Lean Thinking" defines lean as: a new way of thinking to effectively organize human activities, with the goal of eliminating waste and delivering more useful value. The book further summarizes the 5 principles of lean, and also the 5 implementation steps of lean:

1. Define value: Define what value is from the user's perspective and describe it as a specific product or service;
2. Identify value flow: identify and map process steps that create value, and eliminate steps and activities that do not increase user value;
3. Let the value continue to flow: Let the user value flow in the process steps, so that they flow continuously and smoothly to the end user;
4. User value pull: the flow is driven by user value to avoid waste that does not bring user value;
5. Keep improving: Repeat steps 1 to 4 continuously. Pursue perfect value and value flow, and eliminate all waste in the process.

At this level of abstraction, lean thinking has surpassed the manufacturing industry where it was born, and profoundly affected various industries, such as lean government affairs, lean hospitals, lean leadership, lean service industry, lean supply chain, lean education, etc., which also include products Development. In fact, mainstream agile development methods are directly affected by lean thinking and follow the basic principles of lean.

At the same time, as an independent practice system, lean product development is also developing rapidly. It focuses on two goals-the value delivery process and the value itself.

First, focus on the value delivery process. The most representative one is the "Lean Kanban Method", which was developed by David Anderson based on his own practice around 2006, and was systematically summarized in the book "The Kanban Method" published in 2010. "Kanban method" is the specific application of lean thinking in software development. It starts by visualizing the end-to-end value flow of demand delivery, improving the flow efficiency of demand through system practice, and ensuring the quality of the flow process, thereby achieving end-to-end system improvement.

The essential change of this type of lean practice represented by the "Kanban Method" is to shift from focusing on the efficiency of resource use to focusing on the efficiency of value flow. This also drives users from the past local optimization to end-to-end global optimization.

Second, focus on the value itself. The most representative one is "Lean Entrepreneurship". The practice of lean entrepreneurship was originally developed by Steve Blank based on his own and other Silicon Valley entrepreneurial practices. In the book "Lean Entrepreneurship", Eric Ries made a systematic summary of the concept and practice of lean entrepreneurship, and let the concept of lean entrepreneurship Rapidly popularized.

Lean entrepreneurship believes that entrepreneurship is a huge and uncertain process, and its biggest waste is to deliver things that are useless (cannot solve user problems or bring business success). For this reason, it integrates value exploration and discovery into the product delivery process, and proposes the famous "development-measurement-learning" cycle. The cycle starts with the concept to be tested about the market and the product. Next, the first step of the cycle is to develop a Minimal Viable Product (MVP) to verify this concept; the second step: collect feedback from the market and users based on the minimum viable product, and obtain measurement data; third Step: Use data to verify hypotheses, verify or falsify them, and adjust them to generate empirical cognitions. Then, enter the next cycle, continue to explore the business model and product function design.

The impact of lean entrepreneurship far exceeds that of start-up companies. In fact, the book "Lean Entrepreneurship" defines "entrepreneurship" as the creation of new businesses and products in an uncertain environment. “Uncertainty” seems to have become a common feature of the environment in the IT field today. Therefore, the concepts of MVP and the “development-measurement-learning” cycle have become recognized practices in the IT innovation field, and a set of developments has been developed around lean entrepreneurship. Complete innovation practice system, such as lean data analysis, lean customer development, lean delivery design, etc.

Explore and discover effective value, and let the value flow smoothly. Around these two goals, and following lean thinking, lean product development has developed into a systematic practice. The impact of lean thinking on DevOps is also very fundamental, and the three principles of DevOps completely follow lean thinking.

The birth of DevOps

Initially, both the agile and lean communities were only concerned with the practices and behaviors on the development side, and operation and maintenance did not become their focus. However, system development alone is not enough. The developed system must be deployed smoothly in real time and run continuously and stably to realize its value. Traditionally, this part is responsible for operation and maintenance.

From a value perspective, development plus operation and maintenance constitute a relatively complete IT value chain. Of course, a more complete one should also include business. This is something to come, and this is not the focus of the early DevOps community. When DevOps was born, the problem to be solved was the problem between development and operation and maintenance. This is the most prominent problem affecting the IT value chain.

In the traditional IT organization, the demands between the development team (Dev) and the operation and maintenance team (Ops) are different-the development team (especially the agile team) pursues change, and the operation and maintenance team pursues stability. There are often conflicts of interest between the two parties. For example, lean and agile teams take continuous delivery as their goal, while operation and maintenance teams emphasize change control for online stability. The departmental wall is thus established, which of course is not conducive to maximizing the value of IT.

In 2009, at the second Velocity conference held in the United States, John Allspaw and Pauk Hammond from Flickr gave a speech "10+ Deploys Per Day: Dev and Ops Cooperation at Flickr". In this speech, Allspaw and Hammond vividly demonstrated various conflicts between development and operation and maintenance in a role-playing way. There were many golden phrases in the speech, such as "It's not my code, it's your machines!", which deeply reflected the current status of the relationship between Dev and Ops. Then, they showed how to eliminate the barriers between the development team (Dev) and the operation and maintenance team (Ops), and the two sides worked together to make the release and operation of the software continuous and efficient with the help of tools and cultural changes.

This speech is a landmark event in the development of DevOps. It raises the right questions-in order to deliver faster and realize value, the gap between development and operation and maintenance must be bridged, and it provides solutions-in order to bridge the gap between development and operation and maintenance, culture, A series of changes in tools and practices.

In the same year, Patrick Debois, an independent Belgian IT consultant, saw this speech and was inspired to organize the first DevOpsDays. DevOps officially took the stage. The concept of DevOps became popular, and its related tools and practices also developed rapidly. During the period, the emergence of cloud native technologies represented by containerization and automatic orchestration and scheduling also greatly accelerated this process. Today, DevOps has become one of the core capabilities of enterprise digitalization, and it is the basic requirement for IT delivery and operation.

Later, in the two books "Phoenix Project" and "DevOps Practice Guide", Gene Kim and others summarized the three-step work method of DevOps implementation. They are:

Flow principle: Focus on the overall value flow of the IT system, optimize the overall situation, and ensure the rapid flow of value from the left (upstream) to the right (downstream).

Feedback principle: Create a feedback loop from left to right, shorten the feedback period and amplify the feedback effect. In this way, you can respond to and understand internal and external customers faster, and instantly acquire the knowledge needed for improvement.

Principles of continuous experimentation and learning: create a culture of taking risks, experimenting continuously and learning from mistakes, improving abilities through continuous attempts, and improving the resilience of the system.

Kim and others believe that these three-step work methods are the value and philosophical foundation of all other DevOps processes and practices, and all DevOps models can be derived from these three principles.

After a little exploration, you will be able to realize that the DevOps three-step work method is a copy of the lean principle. More precisely, it is a concrete example of lean principles in the context of IT development and operation. In fact, the basic part of DevOps reflects the influence and application of lean principles.

to sum up

Looking back at the development of agile, lean and DevOps, we can draw the following two conclusions.

First, DevOps is a natural evolution of agile development practices. The goal of agile development is to "deliver value earlier and respond to changes more flexibly". The agile movement starts on the development side, but if the operation and maintenance side does not make changes, it will definitely become a bottleneck, and ultimately the goal of agile will not be achieved. In order for agile practices to give full play to their true value, the linkage of development, operation and maintenance is imperative.

Second, DevOps is the inevitable result of the application of lean thinking in the IT field. The goal of lean product development is: "smooth delivery of effective value", and lean thinking requires end-to-end system optimization and continuous improvement. Development and operation and maintenance are two important parts of the system, and neither is indispensable. The three principles of DevOps are concrete examples of lean thinking in the field of IT development, operation and maintenance.

Finally, starting from lean thinking, we can see the inevitable development direction of DevOps, which is to extend to the business side. Business is the source of product development and operation and maintenance, and a complete value stream must start from the source. This is not a prediction, but a fact that is happening. Most DevOps implementations have included the business side and become BizDevOps, but the title of DevOps has been deeply rooted in the hearts of the people. We will continue the DevOps statement, but by default , It includes business.

With the development of DevOps, digital transformation has also become the consensus of the business community. Most of the enterprise digital frameworks regard DevOps as one of the core capabilities, and the scope of DevOps has continued to expand, making it an inevitable choice for companies seeking to improve their digital capabilities. In the next section, we will analyze the fundamental problems to be solved by DevOps in the context of digital transformation.

Free download of "Alibaba DevOps Practice Guide"

It is recommended by Alibaba partners and many industry leaders, He Mian, Chen Xin and other 17 senior Alibaba technical experts jointly produced the product, Alibaba's ten-year DevOps experience precipitation summary, and Alibaba's DevOps implementation practice.

Go to: https://developer.aliyun.com/topic/devops , download the full version of the e-book.

Copyright statement: content of this article is contributed spontaneously by Alibaba Cloud real-name registered users. The copyright belongs to the original author. The Alibaba Cloud Developer Community does not own its copyright and does not assume corresponding legal responsibilities. For specific rules, please refer to the "Alibaba Cloud Developer Community User Service Agreement" and the "Alibaba Cloud Developer Community Intellectual Property Protection Guidelines". If you find suspected plagiarism in this community, fill in the infringement complaint form to report it. Once verified, the community will immediately delete the suspected infringing content.