About DevOps CI/CD Pipeline, just read this article | IDCF

When it comes to DevOps, many people think of CI/CD Pipeline, and even many individuals or companies think that completing CI/CD Pipeline is equivalent to achieving DevOps. Although this view is biased, it reflects the CI/CD Pipeline from the side. DevOps plays a pivotal position. CI/CD are the two key core capabilities of DevOps. The true realization of CI/CD Pipeline will accelerate the transformation of enterprises to DevOps. This article will uncover the mystery of CI/CD Pipeline and take a closer look.

1. What is CI/CD Pipeline

Pipeline refers to a pipeline or assembly line, which is similar to a production line in a factory. Raw materials are input from one end and pass through multiple work centers in the middle. The output of one work center is used as the input of the next work center, and finally high-quality products are output at the other end. CI/CD Pipeline refers to the CI/CD pipeline. The source code is input at the input, and after a predetermined workflow, products that are valuable to users are finally output at the output. It can be seen from the literal that the focus is continuous and the assembly line.

The simple schematic diagram of CI/CD Pipeline is as follows.

After the developer submits the code, it goes through a series of processes to finally produce a valuable application.

2. Why do we need CI/CD Pipeline

The emergence of DevOps is to solve practical problems: to break down the barriers between developers and operations and maintenance personnel. CI/CD Pipeline can help achieve this goal: after the code is submitted, the developer has a CI/CD process to verify the functionality, robustness, security, etc. of the developed code. If the verification fails, it will be returned to the developer. Continue to modify; if the established process is passed, production deployment can be carried out.

Of course, the entire process must be complete, the testing process must be rigorous, and the development environment, testing environment, and production environment should be as consistent as possible to avoid online failures caused by inconsistent environments. And all the changes in the whole process, including code changes, deployment script changes, and environment configuration changes are traceable, so that they can be backtracked and reviewed after problems occur. In this way, developers will not worry that the code cannot be deployed to the production line in time, and the operation and maintenance does not have to worry that the newly launched code will cause the system to crash.

In addition, the CI/CD pipeline helps shorten the application release cycle, increase the release frequency of the application, quickly obtain market feedback, and respond in a timely manner. This virtuous circle helps applications seize the market and bring benefits to enterprises.

The story of the Bill team in "The Phoenix Project: A Legend of IT Operation and Maintenance" is a good proof of the above process: after the Bill team sorted out the deployment process of the Phoenix project, the "pipeline" was carried out (although not in the book) Called CI/CD Pipeline, but the content is the same, which can be considered as the prototype of CI/CD Pipeline) Transformation, while greatly increasing the release frequency of the Phoenix project, the contradiction between the business department and the IT department has become less and less. When the company becomes profitable, the risk of the company being split up and the IT department being outsourced is avoided.

Based on this, CI/CD Pipeline generally has the following characteristics:

• Standardized steps: the construction, testing and other steps in the application delivery pipeline can not be omitted, and they are interlocking.
• Automatic deployment: From the moment the code is submitted, all processes should be automatic. The benefits of automation are undoubtedly: time saving, reducing human intervention, and avoiding human misoperation; "one-click" deployment makes deployment simple.
• Applicable to multiple languages: CI/CD Pipeline should be applicable to most or all language types of applications, java, php, python. It may only be necessary to make a few adjustments for different languages. There is no need for each language to correspond to its own CI/CD Pipeline.
• Portability: When applications are migrated in different deployment environments (such as from public cloud to private cloud, from AWS's Kubernetes platform to IBM's Kubernetes platform), Pipeline can be completed without changes or minor changes Migration, which improves flexibility and reduces workload.

Third, the stages included in the CI/CD Pipeline

Generally speaking, the application delivery pipeline has several stages as shown in the following figure:

3.1 Planning stage

At this stage, the project manager or product manager obtains the relevant information of the application from the user, so as to formulate a development plan to guide the later development. Today, when agile is prevalent, many companies use agile for development management at this stage, and complete the development of user stories in an iterative manner.

3.2 Coding stage

At this stage, developers mainly use the IDE to develop code. Developers use some plug-ins installed in the IDE to write code that meets the company's coding specifications and security requirements.

3.3 Construction phase

The construction phase can be regarded as the true beginning of the CI/CD Pipeline. Developers complete code development locally and submit the code to a source control tool (such as git). At this time, the CI process will be triggered, code scanning, compilation, and unit testing will be performed. And coverage testing and other work. If the process is successful, the code will be merged into the main branch (the merge can be manual or automatic, depending on the development model and regulations of the project). Once a link fails, the entire process will be interrupted and the corresponding information will be fed back to the developer.

3.4 Test phase

The code is deployed to the test environment and a series of tests are carried out, including manual, automatic; functional, performance, and security. It should be noted that the environment required for testing at this stage is best to be consistent with the production environment to avoid online failures caused by inconsistent environments.

3.5 version preparation phase

After the verification in the test phase is successful, the code can be considered to have the ability to go into production, and the version that has passed the test is marked as ready for production, and only manual or automatic operation (depending on the project regulations) is required to push the version to the production line.

3.6 Deployment phase

Deploy the application to the production line. Deployment can be done manually or automatically. At the same time, blue-green deployment and canary deployment can be used to achieve "zero downtime" and safe deployment.

3.7 Maintenance phase

After the application is online, the operation and maintenance team needs to ensure the stability of the application operating environment. It can automatically expand the capacity to cope with the peak when the traffic is high, and shrink the capacity when the traffic is low to reduce resource consumption.

3.8 Monitoring phase

Monitoring should include two aspects: application monitoring and Pipeline monitoring. With the help of monitoring tools, some effective data is obtained and then fed back to the development team or operation and maintenance team to further improve the stability of the application and the Pipeline. safety.

The CI/CD Pipeline is almost equivalent to the application delivery pipeline, and the relationship between its corresponding stages is shown in the figure below. The application delivery process starts at the planning stage, and the CI/CD Pipeline starts at the application development stage.

Fourth, the realization of CI/CD Pipeline

CI/CD Pipeline can also be implemented with a three-step work method:

• Step 1: Realize the forward flow from the input source code to the output end product, linking up the source code construction, testing, version preparation, deployment, and operation and maintenance steps;

• Step 2: Realize continuous feedback from each stage to the development stage;

• Step 3: Cooperate with development and security teams for continuous improvement during operation. For example, visualize the deployment process, inspection reports, and monitoring information.

The implementation of CI/CD Pipeline relies on some tools, which is why many people say that DevOps is an important reason for the collection of tools. Although the view is not comprehensive, tools are indeed an indispensable part of DevOps.

The selection of tools can follow the following principles:

• Embrace open source

Embracing open source has become a new trend in the IT industry. Open source means free, which can further reduce business costs. In addition, open source code is very helpful in tracing the root cause of software problems. The most important point is that you can use Open source to quickly launch valuable commercial products. For example, with the help of open source Linux, Redhat has launched enterprise-level Linux products; with the help of open source Docker and Kubernetes, Amazon, Microsoft, Google, IBM and other companies have accelerated their own enterprise-level products. Commercial use of cloud products.

• Strong community and large number of users

The choice of open source tools must consider the size of the community and the number of users. The community is large and there are many users. Open source tools can iterate quickly and have powerful functions. If problems occur, corresponding solutions can be found. For example, Jenkins, a powerful Chinese and English community, with more than tens of millions of users, has made it the last welcome continuous integration tool.

• Rich API

The CI/CD Pipeline is a collection of tools. The work center and the work center are the interaction between tools and APIs. If a tool has a rich API, it is easier to integrate it into CI/ CD Pipeline. For example, Github has various APIs such as release, commit, and pull request.

• Easy to learn and use

A tool that can be used quickly and easily can accelerate the development of CI/CD Pipeline, reduce the difficulty of CI/CD Pipeline maintenance, and greatly reduce the workload.

V. Development Trend of CI/CD Pipeline

CI/CD Pipeline has been developing and evolving, and its main development changes are as follows:

5.1 Cloud Native

The development direction of CI/CD Pipeline is the same as that of applications: cloud native. There are some cloud-native CI/CD Pipeline tools, such as Jenkins, Tekton, etc., which can further improve the efficiency of CI/CD Pipeline and accelerate the construction of cloud-native applications.

5.2 GitOps and ChatOps

GitOps and ChatOps will play an important role. GitOps and ChatOps can combine teams, tools, processes, automation, etc. into an efficient and transparent workflow. The work progress is clear at a glance, all changes are controllable and traceable, and the product development efficiency will follow the communication and coordination between the teams. Increase and improve.

Github Action is a typical example of GitOps. Slack's bot is a typical example of ChatOps.

5.3 Platformization

The benefits of platformization are unquestionable: through the effective integration of tools and processes, the entire process of the software development life cycle is made transparent, data visualization, and deployment process automation, providing an important basis for continuous feedback and continuous improvement.

Six, some anti-patterns of CI/CD Pipeline

6.1 has nothing to do with development

Developers have insufficient knowledge of Pipeline, and the development model will not be changed. The frequency of code submission has not increased, it may be once every three days, or even once a new feature is submitted. In this way, the advantages of Pipeline cannot be fully utilized, and the frequency of product releases is not high. In addition, the scope of work identified by development is only to the moment the code is submitted, without caring about the success of the overall process, and without obtaining continuous feedback. Once there is a problem, go directly to O&M. "Wear new shoes and go the old way" again and again has led to the failure of DevOps transformation.

6.2 Build time is too long

In order to ensure code quality and achieve one-click deployment, Pipeline will integrate all the steps. Pipeline is extremely bloated. A build takes tens of minutes or even hours (for example, using appscan to complete the dynamic and static scanning of the code). Developers are unwilling to submit code once and wait for such a long time to build. It will reduce the frequency of code submission and submit multiple changes at once. As a result, Pipeline has become a bottleneck in increasing the frequency of product releases, and continuous integration and continuous deployment are even more out of the question. Pipeline should support product deployment anytime, anywhere, and can complete multiple deployment requirements a day. It's like a road. There is no restriction on whether people or cars run on it, how fast and how many.

6.3 Simple feedback

When the failure of a certain step leads to the termination of the entire process, only one sentence of construction failure is used to notify the relevant personnel. This kind of unclear notification requires relevant personnel to work together to investigate the root cause of the problem, which is a waste. The information generated during the build process should be as detailed as possible, such as code compilation failure, unit test failure, and flaws in image detection. If it is a microservice, it should also be clear which service it is. In this way, there will be a corresponding person in charge to quickly repair it at the first time.

6.4 Vulnerabilities are a matter for the security team

When security vulnerabilities are scanned, they are considered to be the scope of the security team's work. The report is there, no one is in charge. In fact, the scope of security is very large. Wrong configuration information and sensitive information in the code are all considered vulnerabilities. Different teams should be responsible for vulnerabilities at different levels. For example, the development team should be responsible for the application code, the operation and maintenance team should be responsible for the infrastructure, and the two teams should coordinate and cooperate with the security team to fix them together. Loopholes.

6.5 Unreasonable manual intervention

When the code coverage for repairing emergency faults does not meet the standard, but has to go online, reduce or even cancel the code coverage test to make the Pipeline successful; when the Pipeline fails in the last step, in order to save time and avoid rebuilding, pass Manually modify the process or configuration to complete the pipeline. This kind of unreasonable manual intervention introduced in order to save time destroys the integrity of the Pipeline. Manual intervention is untraceable, and it is difficult to trace the failures caused by manual intervention. Pipeline is a process. Since it is a process, it should follow the process.

6.6 Fear of failure

Fear of failure to build, failure means that there is a problem with the code, everyone will fry. In order to get a green result every time we build, lower the detection threshold and cut out the steps that often make mistakes. Finally, we develop happy, operate and maintain happy, test happy, and we collectively happy. In fact, failure is the driving force for continuous improvement. The continuous improvement of the pipeline is the practical pipeline.

Seven, concluding remarks

As you can see, "Process + Tools + Automation ≈ CI/CD Pipeline". It is not difficult to understand why DevOps is a process, a collection of tools, and automation and other misunderstandings. DevOps is a cultural movement, and CI/CD Pipeline is a powerful means of implementing this movement in enterprises.

appendix

The following figure is an example of CI/CD Pipeline based on the Kubernetes platform, for reference only.

The tools appearing in the picture are all open source tools, but it does not mean that the tools appearing in the picture are the best and must be selected. The selection of tools should be carried out according to the aforementioned principles.
The bottom part belongs to the monitoring system and notification system, which are used in every environment and run through the entire process. So only draw once.
If the CI process is successful, the test environment can be deployed manually or automatically; similarly, if the test environment is successfully tested, the production environment can be deployed manually or automatically. This completes the CD process.

Author: Ma Jinghe

Ma Jinghe, also known as Xiaoma Ge, is named Xiaoyaozi. He has done LTE 4G network protocol development, and later turned to DevOps, preached on Cloud Native DevSecOps, likes to study Cloud Native related technologies such as docker, kubernetes, istio, etc., is willing to share, and runs the DevOps SIG official account. Held a DevOps Workshop event at the Dalian DevOps community event.

reference

• Author of "Phoenix Project: A Legend of IT Operation and Maintenance" (Gene Kim, Kevin Behr, George Spafford), People's Posts and Telecommunications Press
• Author of "DevOps Implementation Manual Using DevOps in Multi-level IT Enterprises" (Sanjeev Sharma), Tsinghua University Press
• https://opensource.com/article/19/4/devops-pipeline
• https://www.red-gate.com/simple-talk/sysadmin/devops/introduction-to-devops-the-application-delivery-pipeline/
• https://devops.com/continuous-delivery-pipeline/
• https://www.plutora.com/devops-at-scale/pipeline
• https://medium.com/taptuit/the-eight-phases-of-a-devops-pipeline-fda53ec9bba

Source: CIO Talk

Every Thursday in May at 8pm, [Brother Dong has something to say] Quality and testing session. The public account can get the address of "Quality"

• 0506 Zhu Shaomin "How to Maximize Software Testing Effectiveness"
• 0513 Mysterious Guest Mysterious Topic
• 0520 Chen Ji "Yes, going to QA is the most effective way to improve quality!" 》
• 0527 Shi Huibin "Continuous Testing of DevOps Practice"