App Deploy as Code! SAE & Terraform implement IaC-style deployment of applications

Author: Song Yang (Liu Xin)

Enterprise cloud migration is a development boom in recent years. More and more enterprises deploy their applications in various cloud vendors, and take advantage of the elasticity, flexibility, security, and low cost brought by cloud computing to easily help enterprises build their own Applications.

With the development of enterprise scale and business form, an application needs to combine multiple cloud resources in order to provide external services. In order to facilitate the development and debugging of R&D personnel, each application needs to be deployed to multiple environments, and the cloud resources are simply managed manually. This will increase the burden on infrastructure managers.

Therefore, more and more enterprises choose to use code (instead of manual processes) to define infrastructure, that is, IaC (Infrastructure as Code), which treats infrastructure like application software, which not only saves complicated manual operations, but also utilizes The versioning and abstraction capabilities brought by the code configuration natively.

In the era of cloud computing, application deployment often relies on multiple infrastructures (computing, storage, and network). Enterprises often need a PaaS platform to deliver their own applications, but the development and maintenance of the PaaS platform requires a large investment in technology and manpower. For fast-growing companies, it is more desirable to focus on the core business of the company.

Alibaba Cloud's Serverless App Engine SAE (Serverless App Engine) is an application-oriented Serverless PaaS platform that naturally supports creation and management through IaC. Terraform, as the de facto standard in the IaC field, has become the preferred tool for enterprise IaC management. Combining Terraform and SAE will have some wonderful chemistry, and enterprises can manage enterprise applications through simple code configuration files in an ADaC (Application Deploy as Code) way.

Below, we briefly introduce the great convenience brought by IaC and SAE to enterprises, and feel the dimensionality reduction blow brought by SAE & Terraform to traditional enterprise IT facility management through an example of using Terraform to create an SAE application.

Infrastructure as Code

Enterprise Infrastructure Management Development History

The normal external services of applications need to rely on basic resources such as computing, storage, and network. They are the underlying context for the application to function properly. These resources are also known as environmental infrastructure. Under the traditional management model, most companies will have a dedicated operation and maintenance team to manage their formal production and test environments. With the upgrade of the business and the growth of the company's scale, the operation and maintenance team will go through about 3 stages in basic resource management:

• Manual operation and maintenance : In the early stage of enterprise development, the business type and scale of the enterprise are in the initial stage, and the basic environment is a resource with a relatively low frequency of change. The operation and maintenance team can often build the basic resources required for the service through manual management. Enterprises using cloud services can create cloud resources through mouse operations on the cloud vendor's console. At this stage, the infrastructure requirements of R&D personnel are very low, and operation and maintenance personnel can meet it through manual operation and maintenance.
• Scripted operation and maintenance operations : With the development of enterprise scale and business, the operation and maintenance team will receive more and more environment creation requests. Most members of the operation and maintenance team will spontaneously write standardized documents and scripts to improve the efficiency of environment creation. , or through the CLI to assist the creation of resources, but because a service often requires the cooperation of multiple infrastructure resources to provide services to the outside world, the script cannot simply handle the dependencies between different infrastructures. This section of the operation and maintenance personnel creates the environment It will gradually become a tedious and inefficient work.
• Infrastructure as code : Manual operation and maintenance and document scripting have greatly affected development efficiency. Enterprises will gradually abstract the infrastructure into code, configure the infrastructure in a way of managing the code, and perform version control and rollback on the environment infrastructure just like the code, and the same code modules can be reused between multiple environments to achieve Rapid delivery of environmental infrastructure.

Terraform came into being

In 2014, HashiCorp launched Terraform, a tool for safely and efficiently building, changing, and versioning infrastructure, and Terraform is the king of IaC today. Using Terraform to manage your enterprise infrastructure can bring a variety of benefits to your business:

• Use declarative IaC to manage infrastructure : Declarative description can ensure that a consistent state can be achieved even if code is executed multiple times. Using code to describe basic resources can more vividly show the differences between different environments. No matter what environment goes wrong, a new environment can be quickly reproduced.
• Rich modules ecosystem : Cloud resources of almost all cloud vendors are included, and users can use various official and community-provided high-quality modules in Terraform Registry, an officially maintained module repository. So that users do not need to repeatedly write modules of other cloud vendors, and use the capabilities of the open source community to continuously improve and expand the Terraform ecosystem.
• Resource dependency management : Terraform builds a DAG topology diagram of all resources based on the definitions in the template. For resources with dependent resources, they will be executed in order according to the dependencies, and resources without any dependencies will be created in parallel to ensure efficient execution.

Enterprise Application Deployment in Cloud Computing Era

Application deployment often involves VPC network management and division, creation of virtual machines, and exposure of application service addresses through load balancing. Today, when the microservice architecture is prevalent, enterprises also need to deploy and operate some microservice components to provide functions such as service discovery, configuration management, and lossless online and offline to ensure that applications can provide stable and continuous services. In order to monitor the running status of the application, it is an essential part to understand the health status of the application through information such as Trace, Metrics, and Logs.

A healthy application requires the ability to combine a series of infrastructures, and the testing and launch of each application will consume a lot of energy from R&D and operation and maintenance managers. In fact, it is not difficult to find that there are similarities in the dependent resources of each application, and many differences are only reflected in some configuration items.

If a product can combine the capabilities of these infrastructures and provide external application concepts, R&D personnel only need to care about the writing of business codes, and operation and maintenance managers do not need to manage and maintain a large amount of infrastructure, and the R&D efficiency of enterprises will be greatly improved. .

The serverless application engine SAE provided by Alibaba Cloud is such a product that reduces the mentality of enterprise IT personnel.

SAE is an application-oriented Serverless PaaS platform, which can help PaaS layer users avoid IaaS operation and maintenance, use on-demand, and charge by volume, so that low-threshold microservice applications can be migrated to the cloud. Compared with other serverless products, it abstracts the concept of applications, helps enterprises shield the creation and management of a large number of infrastructures, and provides a complete set of microservice solutions to support the development of mainstream microservices such as Spring Cloud, Dubbo, and HSF. The framework realizes the perfect combination of serverless architecture and microservice architecture.

SAE provides nanny-level hosting services. Developers only need to provide a compiled JAR package or WAR package to deploy an application with a full set of microservice experiences, and infrastructure managers do not need to manage a large amount of infrastructure.

SAE & Terraform, Application as Code

Now you can create and manage applications on SAE through Terraform, combining Terraform IaC and SAE's application-centric capabilities, so that enterprises can code all the configuration of the application, easily describe and manage the application, when due to human or other accidents Factors that cause the application to be in an unhealthy or unusable state, we can quickly fork an identical application to minimize the impact.

Terraform describes the infrastructure through a declarative HCL language. The programmer tells Terraform the resource status I expect to get, and Terraform does the rest to create it. But Terraform doesn't care about the running state of the created application. The normal operation of the services running on Terraform to create resources needs to be guaranteed by developers. The bottom layer of SAE is based on Kubernetes. Using the declarative capabilities of Kubernetes, SAE can ensure that application services remain in the declared state and provide services normally. Terraform combined with SAE better utilizes the ability of declarative description of application resources.

Below, please follow our pace and feel the charm of IaC together.

Preparation

This section shows you how to use Terraform's IaC and dependency management capabilities to quickly pull up applications in different environments and combine SAE and other cloud resources to build your applications.

Before we start, let's clone the demo-related code locally:

 git clone [e-mail]:yangsoon/terraform-sae.git

Expose the AK and SK necessary for creating cloud resources to environment variables. (When Terraform creates cloud resources, it will use environment variables to specify AK and SK to create cloud resources).

 export ALICLOUD_ACCESS_KEY=(your access key id)
export ALICLOUD_SECRET_KEY=(your secret access key)

Quickly create multi-environment applications

Go to the project's root directory and briefly look at the project's directory structure. The modules folder contains the packaged environment infrastructure modules, including the lb that uses Alibaba Cloud SLB to achieve load balancing and external network access capabilities, the network module that provides the private network, and the webserver module that further encapsulates the Alibaba Cloud SAE resources, which is convenient for When users create applications in different environments, they can be directly reused to reduce unnecessary code copying.

stage and prod store the resource configuration of the enterprise in the staging environment and the production environment respectively. The environment infrastructure of the pre-release environment and the production environment are quite different, and the resource configuration of the production environment has higher security requirements. In order to prevent the resources of the production environment from being damaged due to misoperation, we isolate them through the file layout.

Open the ./stage/webserver/main.tf file and you can see that in the staging environment we specified to use the webserver component to create the application.

 module "network" {
  source   = "../../modules/network"
  vpc_name = var.vpc_name
}

module "webserver" {
  source = "../../modules/webserver"

  sg_id      = module.network.SG_ID
  vpc_id     = module.network.VPC_ID
  vswitch_id = module.network.VSWITCH_ID

  app_name       = var.app_name
  image_url      = var.image_url
  namespace_name = var.namespace_name
  namespace_id   = var.namespace_id
}

Fill in the application name and mirror address corresponding to the pre-release environment in ./stage/webserver/vars.tf (here, nginx is used as an example).

 variable "app_name" {
  description = "The name of the application"
  type        = string
  default     = "webserver-stage"
}

variable "image_url" {
  description = "The image of the application"
  type        = string
  default     = "nginx:stable"
}

Next, we use Terraform to quickly build a pre-release environment.

1. Go to the ./stage/webserver folder and initialize the Terraform workspace

 cd terraform-sae/stage/webserver
terraform init

This step will help you initialize the Terraform submodule and install the necessary plug-ins. After successful execution, the following information will be displayed

2. View the type and number of resources created by the pre-release environment

 terraform plan

After executing terraform plan, there is a lot of output content, and some information is intercepted here. You can see that 6 resources will be created in the pre-release environment, and the output result will display the specific configuration information of the newly created resources.

3. After confirming that it is correct, we start to create the resources required for the staging environment.

 terraform apply

Terraform apply will show you the resource information created by this execution again. After confirming that it is correct, enter yes, then Terraform will actually create the resource for you.

Wait for a while here and wait for the resource creation to complete. You can log in to the Alibaba Cloud console to view the application you just created.

After a series of debugging tests have passed the verification, you can continue to create resources for the production environment.

Open the ./prod/webserver/main.tf file, you can see that we can directly reuse the webserver module, and modify some application names and application images to the configuration related to the production environment. In addition, we have created a new SLB , allowing the application to be accessible from the outside world. Next, we continue to create the application of the production environment.

 module "lb" {
  source = "../../modules/lb"

  slb_name     = var.app_name
  address_type = "internet"
  vswitch_id   = module.network.VSWITCH_ID
}

resource "alicloud_sae_load_balancer_internet" "example" {
  app_id          = module.webserver.app_id
  internet_slb_id = module.lb.slb_id
  internet {
    protocol    = "HTTP"
    port        = var.port
    target_port = 80
  }
}

module "webserver" {
  source = "../../modules/webserver"

  sg_id      = module.network.SG_ID
  vpc_id     = module.network.VPC_ID
  vswitch_id = module.network.VSWITCH_ID

  app_name       = var.app_name
  image_url      = var.image_url
  namespace_name = var.namespace_name
  namespace_id   = var.namespace_id
}

Modify ./prod/webserver/vars.tf and fill in the application name and mirror address corresponding to the production environment.

5. Go to the ./prod/webserver folder and initialize the Terraform workspace

 cd terraform-sae/prod/webserver
terraform init

6. View the type and number of resources that will be created in the production environment

 terraform plan

Here, the production environment will create 2 more resources related to SLB.

7. After confirming that it is correct, we start to create the resources required for the production environment.

 terraform apply

After waiting for a while, we created an application that can be accessed from the external network. The output contains the external network address we can access. By requesting this address, you can see that an nginx server has been set up successfully.

So far, we have leveraged Terraform's capabilities to quickly create multi-environment applications.

Efficiently combine SAE and other cloud resources using Terraform's dependency management capabilities

SAE also has higher-level gameplay in the Terraform ecosystem, and you can combine other cloud resources arbitrarily. Taking RDS as an example, you can inject some connection information of RDS into the SAE application in the form of environment variables. After the application is started, you can connect to the target database through the information of environment variables.

Open ./prod/webserver-with-db/main.tf, we introduced the mysql module to help us create an Alibaba Cloud RDS instance, and the database connection information after RDS is created is injected into the SAE application in the form of environment variables.

 module "mysql" {
  source = "../../modules/mysql"

  databases = [
    {
      "name" : "sae-demo",
      "character_set" : "utf8",
      "description" : "sae demo database"
    },
  ]

  rds_instance_name = var.rds_instance_name
  rds_account_name  = var.rds_account_name
  rds_password      = var.rds_password
}


module "webserver" {
  source = "../../modules/webserver"

  sg_id      = module.network.SG_ID
  vpc_id     = module.network.VPC_ID
  vswitch_id = module.network.VSWITCH_ID

  app_name       = var.app_name
  image_url      = var.image_url
  namespace_name = var.namespace_name
  namespace_id   = var.namespace_id

  envs = [{
    name  = "DB_HOST"
    value = module.mysql.DB_HOST
    }, {
    name  = "DB_PORT"
    value = module.mysql.DB_PORT
    }, {
    name  = "DB_PASSWORD"
    value = module.mysql.DB_PASSWORD
    }, {
    name  = "DATABASE_NAME"
    value = module.mysql.DATABASE_NAME
  }]
}

Let's demonstrate creating an application that uses RDS as a data store.

1. Similar to the previous two examples, we enter the terraform-sae/prod/webserver-with-db directory to perform workspace initialization.

 cd terraform-sae/prod/webserver-with-db
terraform init

2. Start creating resources

 terraform apply

After the resource is created, the public network address of the database will be output.

3. Check environment variables in SAE applications

You can see that the connection information of the created RDS has been configured in the environment variables.

After executing the Demo command to create cloud resources, remember to destroy the created demo resources in time to avoid unnecessary deductions.

 $ cd terraform-sae/stage/webserver && terraform destroy
$ cd terraform-sae/prod/webserver && terraform destroy
$ cd terraform-sae/prod/webserver-with-db && terraform destroy

Summarize

The combination of SAE and Terraform can help enterprises manage their applications like code, and operations on resources can be audited, traceable, and rollbackable, and at the same time reduce the risk of human operations. SAE abstracts the concept of applications, helps enterprises shield the creation and management of a large number of environmental infrastructures, lowers the threshold for users, and helps enterprises quickly migrate to the cloud.

references

[1] Yevgeniy Brikman. Terraform: Up & Running: Writing Infrastructure as Code. O'Reilly Media

[2] Qiao Liang. "Continuous Delivery 2.0". People's Posts and Telecommunications Press

---

Pay attention to the Alibaba Cloud Cloud Native Official Account, reply to the keyword [Feitian Cloud Native] in the background, and download the most complete information on the summit!