ChatOps practice in large-scale applications based on CODING CD + Nocalhost

The author of this article: Hongya Technology CTO-Lu Xingmin

Hongya Technology focuses on the development of information technology and provides high-quality teaching services for information technology-related majors

background

ChatOps originated from GitHub. It is centered on the communication platform. Through dialogue and interaction with robots, developers can complete the work carried by DevOps only in the chat window.

The IT training and teaching platform "Green Pepper Class" serving 600+ colleges and universities, why choose ChatOps to carry the business, and how to combine SaaS tools with open source tools to form a complete technical solution? This article will reveal the answer for you.

Why does the "complex application" development and testing phase require ChatOps

• With the introduction of deployment tools and deployment pipeline processes into the entire development iteration process, the deployment of development and test environments using the release list mode often requires multiple Web consoles to be opened, which is more cumbersome for daily development iterations.
• With the development of the project, there will be multiple git repo in the project, and each git repo will generate multiple products for deployment. The method based on manual selection is very unfriendly for developers to develop and test.
• In terms of message notification, although Webhook is used to notify the project collaboration information in a group, all notifications of the project are sent to a group, causing information explosion and gradually losing the meaning of notification.

First of all, let’s look at the steps required for our team’s current deployment process. We need to go through the processes of "waiting for the success of CI construction", "selecting the required products and environment on the release list", and "deployment". Among them, the selection of products needs to be selected every time it is released, and it is particularly cumbersome when there are many components.

Not all scenarios require ChatOps. The emphasis here is on "complex applications" because after the application complexity increases, it will face complex configurations, complex products, and complex processes. Therefore, ChatOps tools are needed to reduce the development and testing process. The deployment difficulty is . For simple applications, such as a single application in the initial stage of a project, there is no need to spend a lot of trouble and complicated tool processes. It is very efficient to integrate a small part of the process of automatically updating the test environment in CI.

How to combine CI/CD system and IM open platform to build ChatOps tool

The current status of CI/CD implementation and selection considerations

Continuous integration

Continuous integration is the foundation of all processes, and the goal is also very clear, which is to unify the build environment and product types to facilitate subsequent deployment and use. In the context of the current containerization process, we have also selected container images as the final product, and the developer output is uniformly container images, which is convenient for deployment. All code repositories, whether complex or not, will create a Jenkinsfile for construction.

Application definition selection

In the choice of application definition, after experiencing the initial PaaS platform custom application model and code warehouse to store static Manifest files, Helm was finally selected as the application definition tool, mainly based on the following considerations:

• The deployment method is simple and can be installed directly with a single command. can be deployed and upgraded with one command even if it is separated from the deployment tool in a privatized environment where tools are relatively scarce.
• Use a template for definition, which is convenient for multiple copy deployment and differentiated configuration.
• The Helm chart is stored through the product library, the dev environment uses the build number for version push, and the production environment is tagged through the Helm warehouse for version push to realize the versioning of the "application definition".

Application deployment tool selection

The CODING CD is selected on the application deployment tool, which is mainly based on the following considerations:

• Separation of application definition and component version.
• Load common configuration based on environment.
• Publish startup parameter customization.

The Helm chart and container image are input as products, and the Helm chart version and the image version are separated through product binding to realize the independent configuration of application definition and application component version.

uses the values file reference to facilitate unified configuration of "a certain type of" environment, such as the differentiated configuration between different vendors of public cloud.

Build a ChatOps system suitable for the team

The goals of ChatOps tool building

• Solve the problem of messy messages, classify messages with project iteration as the granularity, and create IM groups.
• Solve the problem of cumbersome creation of the development and test environment and the need for verbal agreement, and create an independent test environment with project iteration as the granularity.
• Try to avoid web console operations.
• After the iteration, the environment and group are automatically cleaned up.

The development and testing process flow combing

As shown in the figure below, we sort out the entire deployment process of development and testing. Among most cumbersome part of 16125c04ea0e21 that requires multiple manual operations is the process of "deployment configuration" + "version selection". , how to update the product to the corresponding environment according to certain rules, and remember the current selection is The key to this process.

First of all, we need to template the entire deployment process. Here we use Namespace as the isolation between environments. The two most critical factors in the environment, Namespace and access domain name, are used as startup parameters to template a single deployment pipeline.

Notification quarantine

By taking over the Webhook event, the original project collaborative notification is redistributed. When iterative creation is generated in the project collaboration tool, it automatically triggers the creation of a group of pre-built DevOps robots, and uses IM to optimize the message and increase the convenient entrance. When the collaborative items of the project are changed, automatically added or deleted to the members of the group. At the same time, the environment is also associated with the current iteration and can be quickly created through chat instructions when needed. At the end of the iteration, the recovery group, test environment, etc., are cleaned up.

Currently ChatOps mainly implements the following commands:

• deploy —— Call up the deployment setting card.
• branch —— Set the branch corresponding to a certain warehouse, find the corresponding product and call up the deployment card.

When the environment is successfully created, the ChatOps controller will record the product selection of the current environment. When the corresponding product is updated, it will automatically update the current environment, so that the test environment can be configured once and automatically updated throughout the iteration.

How to quickly debug the application during the development and testing phase

In the daily development process, the deployment and update of the environment based on the above ChatOps process can meet most of the needs. After the code is pushed, the environment can also be updated in minutes.

When the problems encountered during joint debugging and testing need to be modified, the process of waiting for a CI/CD is very long. In addition, when developing new functions and new components, it is also more cumbersome to quickly put them into the test environment. So we are looking for a tool to quickly debug the development environment.

In the early years of server-side development, we often used the sftp plug-in to synchronize local code to the server for execution, then Nocalhost is a containerized rsync tool. When Nocalhost enters the debugging mode, it replaces the corresponding Container image with the specified development image, and adds a file-synchronized Sidecar, which can synchronize the local code to the container, and debug directly for script-type languages, such as Golang. The compiled type can be built locally for synchronization, or built directly in the container.

The key step that needs to be paid attention to during the whole use process is to make an image suitable for development and debugging. Nocalhost provides development images for common environments, but when it is applied to its own team, the content contained in the image is often related to components. Need to customize a development mirror suitable for the current business. Mainly based on the following considerations:

• Try to include the tools used in the mirror and common debugging tools in the actual environment.
• Remove caching components that affect debugging, such as OPcache in PHP.

Summarize

As the complexity of the business increases, there will be more and more repetitive and cumbersome operations in the development and testing process. Building a ChatOps tool based on the existing CI/CD system is an idea to solve this problem. Choose a solution that suits your team. Is the most important.

Click to immediately start the efficient cloud R&D workflow