Plug in Terraform wings for Serverless Devs, decouple code and infrastructure, and achieve enterprise-level multi-environment deployment (Part 2)

In the last article, " Put Terraform Wings on Serverless Devs and Implement Enterprise-Level Multi-Environment Deployment (Part 1) ", I mainly introduced the use of Serverless Devs' multi-environment function. Users may have some questions after reading it. This article will discuss some common problems. to answer.

The relationship between Serverless Devs and Terraform

Some users may ask, since you have already supported Terraform, what role does Serverless Devs have? Is it enough to use Terraform directly?

The positioning of Serverless Devs and Terraform is still significantly different. Serverless Devs is oriented to application management and DevOps, and Terraform is oriented to cloud resources. They are two different fields, but it does not mean that there is no intersection or integration at some levels. The ability to integrate and be integrated is a measure of whether open source tools are standardized. .

Terraform solves the provisioning of cloud resources, which has a very clear methodology. And Serverless Devs should emphasize how to make good use of cloud resources. The relationship between the two can be illustrated in several scenarios:

• Serverless Devs pay more attention to how to upload code or installation dependent shards to NAS, and pay less attention to how to create VPC/switch/security group/NAS mount point;
• Serverless Devs pay more attention to how to upload files to OSS and automatically trigger functions to complete report generation, and pay less attention to how to create OSS Bucket;
• Serverless Devs pay more attention to how to build code/image, make layers, deploy code, release version, and build a complete CI/CD experience, and pay less attention to how to create FC network, log warehouse, and ACR instance;
• Serverless Devs pay more attention to how to debug code remotely, how to log in to online instances, and how to quickly discover business anomalies through logs and monitoring;

It can be seen that Serverless Devs pay more attention to the operation of application running state and operation and maintenance state, which is also the most important mission of the tools of serverless architecture, but Serverless Devs is responsible for the whole life cycle management of serverless applications, and it is inevitable to manage resources. In the process of practice, we found that whether using cloud product SDK or GPLs such as Pulumi requires a lot of energy to connect the resource life cycle, which is very inefficient for component developers to connect more cloud products. Terraform is the most professional in this regard. Whether it is the degree of standardization, the degree of recognition, and the richness of resources, it can well meet the needs of end users and developers. Therefore, the idea of combining Serverless Devs and Terraform was triggered.

Serverless Devs is not coupled with Terraform. On the contrary, Terraform's HCL language is naturally played in the component specification of Serverless Devs, which can be considered as a capability of Serverless Devs to support multiple languages. The value to developers is that they can complete the infrastructure construction with relatively low code, and devote their energy to the development related to the life cycle management of serverless applications. Otherwise, they have to develop a large amount of resource CRUD code, which is very inefficient.

What is the difference between the multi-environment function and using Terraform directly

Since Terraform is also used for multi-environment deployment, what is the difference between deploying resources directly with Terraform?

• Terraform is a personal version of the tool, which requires local management of ak/sk and local installation of Provider; and multi-environment is a multi-tenant service, which does not require users to maintain these;
• The important thing about the multi-environment function is the ability to "manage". For example, the template has version management capabilities. When a new version of the template is released and the IaC changes are incompatible, if the user updates the environment, it will cause unknown problems. In this case, the system It will automatically identify and ensure that changes to the stock environment still use the old version, and will not be affected by incompatible changes;
• Terraform is a purely resource-oriented orchestration tool, which is weakly related to applications; while environments, services, and pipelines can naturally form connections. For example, through the environment, you can perceive which services are used by resources, and services can be obtained through the authorization of the environment. Access to resources, the ability to deploy services to all environments at once in the pipeline, which Terraform cannot do;
• Terraform is just a technology selection for implementing IaC in multiple environments. In the future, it plans to connect with IaC projects such as ROS and Pulumi.

The relationship between multiple environments and environment variables

Use environment variables in CI/CD, configure VPC, NAS, etc. in environment variables. It seems that you can reference environment variables in s.yaml. Why create an environment concept?

Environment and environment variables can be distinguished from the difference in positioning from their names. An environment variable is a set of static configurations. Although some resource configurations can be written into environment variables and referenced in the CI/CD pipeline, this method does not have the ability to store resources. ability to manage.

The environment is an entity resource and has the ability to manage the life cycle of the infrastructure. Through the environment, the addition, deletion, modification and inspection of the infrastructure can be completed, and the user can be granted operation rights through access control. When updating the environment, it can also connect to some security checks. .

The environment can protect the infrastructure. For example, when multiple services share the environment, if the environment deletion is initiated, the system will automatically find that the environment is depended on by other services, and the deletion will be rejected at this time.

Is it only for business users? How do individual developers use it?

I'm a personal developer, I don't know Terraform, and the various template definitions in the article are a bit dizzy. Is it still suitable for me to use this function?

The same applies to individual developers, but this part of users should not be responsible for writing templates. Instead, the platform provides templates for various business scenarios. Developers can use them out of the box. This is also our main follow-up work.

For individual users, some of the most complicated concepts in Alibaba Cloud are RAM, VPC, ECS, SLB, and NAS, and the learning curve is too long. This problem is particularly obvious under the serverless architecture. Serverless claims to have low threshold, low cost, and low operation and maintenance. However, to get started with serverless, you need to understand a lot of concepts and configure a lot of things. Many users are "dissuaded" in the process. , while environment templates and environments can greatly simplify the onboarding costs of cloud products while operating securely. For example, if a user selects a template deployment environment, they can pull up all cloud resources with one click, which is truly serverless.

Implementation principle

• Follow the Serverless Devs component development specification, and implement a component to complete the docking with the back-end service
• The back-end service adopts the architecture of Serverless + K8s, and the function is triggered by the message to complete the rendering of the template and the execution of the deployment task.
• Using KubeVela [ 1] to complete the management of K8s resources and the execution of Terraform tasks

Why multi-environment is a component-level capability and not a CLI capability

Serverless Devs are divided into CLI [ 2] and components [ 3] :

• CLI provides the most general capabilities and does not depend on any components, such as: s init, s config, s verify, --template, --debug
• Components provide specific functions, such as s deploy, s build, s invoke, these are the capabilities of fc components

From the perspective of the versatility of the env command and the problems to be solved, it is also appropriate to do it in the CLI. However, from the implementation point of view, because a server-side control plane is required to complete the deployment of user resources, and for security reasons, it must be completed by a specific cloud service, so it is completed through a component.

Reference link:

[1] KubeVela:

https://kubevela.io/

[2] CLI:

https://docs.serverless-devs.com/serverless-devs/command/readme

[3] Alibaba Cloud function computing components:

https://docs.serverless-devs.com/fc/readme

past review

Add Terraform wings to Serverless Devs to achieve enterprise-level multi-environment deployment (Part 1)

---

Get started with Serverless quickly

In order to allow developers to quickly locate serverless development problems and find corresponding solutions, the Alibaba Cloud Cloud Native Serverless Team has launched the 2022 "Serverless Development Quick Check Manual" , which is currently available for download. We hope to provide Serverless developers with a book that can be quickly checked and understood. It really helps developers quickly solve practical problems encountered in serverless development, so that everyone can enjoy the technical dividends brought by serverless! Click to read the original text and download the manual immediately!