Gluing the Terraform ecosystem to the Kubernetes world

Author: Zeng Qingguo Sun Jianbo

background

With the expansion of product layouts of major cloud vendors, basic computing facilities, middleware services, big data/AI services, and application operation and maintenance management services can be directly used by enterprises and developers. We have noticed that there are also many enterprises building their own enterprise infrastructure mid-stage based on the services of different cloud vendors. In order to manage cloud services more efficiently and uniformly, the idea of IaC has become popular in recent years, and Terrafrom has been successfully adopted and supported by almost all cloud vendors. The cloud service IaC ecosystem with the Terrafrom model as the core has been formed. However, when Kubernetes is popular today, IaC has a wider imagination space. If Terraform IaC capabilities and ecological achievements are integrated into the Kubernetes world, we believe that this is a powerful alliance.

Reason 1: Build a unified enterprise hybrid cloud PaaS platform

At present, most enterprises build PaaS platforms or infrastructure management platforms based on Kubernetes services, and integrate cloud and self-built infrastructure in a unified manner. But in addition to providing infrastructure, various middleware, big data services, AI services, and application observability are also key products provided by cloud vendors. Enterprise platforms need to have the ability to create and destroy more cloud services. Terraform will enter the platform builder's line of sight, so do they need to do docking development outside of Kubernetes again? At the same time, it is also necessary to consider the requirements of continuous release, GitOps, grayscale release, etc. Obviously, it is a better option to complete the docking directly based on Kubernetes.

Reason 2: Create a Serverless Experience for Developers

The essence or goal of cloud computing is serverless. However, the self-built infrastructure is always limited, and seamless access to cloud services can open up an "almost unlimited" resource pool. At the same time, when developers build business applications, in addition to directly completing business service deployment on the serverless platform, they also need to directly obtain services such as message middleware and databases. More enterprises will adopt the solution of docking with cloud vendors, and the operation and maintenance management cost is lower. But for developers, it's best to be transparent and consistent.

Reason Three: More Thorough IaC Capabilities

Everything is a service, and we need a unified model to describe cloud resources, self-built infrastructure, and various enterprise applications. Terraform and Kubernetes can be integrated and unified into an IaC specification for developers. The benefits are the rich operation and maintenance capabilities of managing cloud resources and container ecology at the same time, as well as unified orchestration for complex applications.

Reason 4: The open source version of Terraform works in client mode and cannot be maintained in the final state like Kubernetes.

The Terraform open source distribution can only work in client mode, that is, the user cannot maintain the service state after one delivery, and manual retry processing is required if the delivery fails due to network failures. Kubernetes brings the final state-oriented IaC idea to developers, and maintains the state of target resources through the controller mode, which further enhances the advantages of Terraform tools at the automation level.

KubeVela is a modern software delivery control plane that provides developers with a unified API abstraction, enabling developers to use the same IaC approach to deliver common applications and cloud services at the same time. KubeVela directly supports Terraform API and Kubernetes API downwards, and can reuse all Terraform modules and all Kubernetes objects without modification. With KubeVela, you can easily implement the three appeals of appeal. We have also seen another model of Crossplane project, which is connected to cloud services by defining Kubernetes native CRD to make the experience more native. KubeVela also naturally supports Crossplane API.

Next, let's take a detailed look at how KubeVela applies Terraform to provide users with a unified IaC experience in two parts.

• Part.1 will introduce how to glue Terraform with KubeVela, which requires some basic knowledge of Terraform and KubeVela.
• Part.2 will introduce a practical case of KubeVela delivering cloud services, including:
• • 1) Provide a Cloud ECS instance with a public IP through KubeVela;
  • 2) Use an ECS instance as a tunnel server to provide public access to any container service in the intranet environment.

Convert Terraform modules to KubeVela components

Prepare Terraform Module

If you already have a well-tested Terraform module, you can skip this step.

Before you start, make sure you have:

• Install Terraform CLI [ 1 ]
• Prepare a cloud service account (AK/SK). The use case in this article uses Alibaba Cloud.
• Learn some basics of using Terraform.

Here is the Terraform module I used for this demo [ 2]

1. Download the Terraform module.

 git clone https://github.com/wonderflow/terraform-alicloud-ecs-instance.git
cd terraform-alicloud-ecs-instance

2. Initialize and download the latest stable version of Alibaba Cloud Provider.

 terraform init

3. Configure Alibaba Cloud authorized account information.

 export ALICLOUD_ACCESS_KEY="your-accesskey-id"
export ALICLOUD_SECRET_KEY="your-accesskey-secret"
export ALICLOUD_REGION="your-region-id"

You can also configure account information by creating a provider.tf file.

 provider "alicloud" {
    access_key  = "your-accesskey-id"
    secret_key   = "your-accesskey-secret"
    region           = "cn-hangzhou"
}

4. Test whether the resource creation is normal.

 terraform apply -var-file=test/test.tfvars

5. Destroy all created resources after the test is OK.

 terraform destroy  -var-file=test/test.tfvars

At this point, you can also push this module to your own code repository as needed.

Convert Terrafrom module as KubeVela extension component type

This step is the core. Before starting, please make sure that you have installed the Kubevela control plane [ 3] . If you don't have a Kubernetes cluster, don't worry, it is enough to complete the one-click installation through VelaD for a quick demonstration.

We'll use the Terraform module we just prepared to do the following.

1. Generate KubeVela component definitions.

 vela def init ecs --type component --provider alibaba --desc "Terraform configuration for Alibaba Cloud Elastic Compute Service" --git https://github.com/wonderflow/terraform-alicloud-ecs-instance.git > alibaba-ecs-def.yaml
vela kube apply -f alibaba-ecs-def.yaml

If you have customized Module, please use your own code repository address directly.

So far you have successfully added the ECS module as an extension component type of KubeVela, you can learn more details from here [ 4] . Developers on the Vela platform can start using this type of component directly. You can view the auto-generated component usage documentation with the following command:

 vela show alibaba-ecs
# OR
vela show alibaba-ecs --web

Is it very simple? KubeVela has a complete toolchain for Terraform, but you don't have to worry about repeating this operation for all Terraform modules, because the community has provided users with out-of-the-box plug-ins, and you only need to install the corresponding cloud vendor's plug-in to get it Components that have been converted. Let's use this ability to experiment with some interesting scenarios.

Expose local container applications to the public network using cloud services

In this section, we'll introduce a solution where you can expose any Kubernetes service using a specific port. The solution consists of the following:

1. The KubeVela environment, if you practiced in Part 1, you already have it.
2. Alibaba Cloud ECS, KubeVela will automatically create a tiny ECS (1u1g) through the Access Key.
3. FRP [ 5] , KubeVela will start this proxy on server side and client side.

Prepare the KubeVela environment

• Install KubeVela

 curl -fsSl https://static.kubevela.net/script/install-velad.sh | bash
velad install

See document 1 [ 6] for more installation details.

• Enable Terraform Addon and Alibaba Provider

 vela addon enable terraform
vela addon enable terraform-alibaba

• Add authorization information

 vela provider add terraform-alibaba --ALICLOUD_ACCESS_KEY <"your-accesskey-id"> --ALICLOUD_SECRET_KEY "your-accesskey-secret" --ALICLOUD_REGION <your-region> --name terraform-alibaba-default

Check out Document 2 [ 7] for more details on other clouds.

Deploy an ECS instance with a public IP address and start the FRP service

 cat <<EOF | vela up -f -
# YAML begins
apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
  name: ecs-demo
spec:
  components:
    - name: ecs-demo
      type: alibaba-ecs
      properties:
        providerRef:
          name: terraform-alibaba-default
        writeConnectionSecretToRef:
          name: outputs-ecs          
        name: "test-terraform-vela-123"
        instance_type: "ecs.n1.tiny"
        host_name: "test-terraform-vela"
        password: "Test-123456!"
        internet_max_bandwidth_out: "10"
        associate_public_ip_address: "true"
        instance_charge_type: "PostPaid"
        user_data_url: "https://raw.githubusercontent.com/wonderflow/terraform-alicloud-ecs-instance/master/frp.sh"
        ports:
        - 8080
        - 8081
        - 8082
        - 8083
        - 9090
        - 9091
        - 9092
        tags:
          created_by: "Terraform-of-KubeVela"
          created_from: "module-tf-alicloud-ecs-instance"
# YAML ends
EOF

This application definition will deploy an ECS instance with a public IP address.

You can learn more about each field description with the following commands:

 vela show alibaba-ecs

After executing the above deployment command, you can view the application deployment status in the following ways:

 vela status ecs-demo
vela logs ecs-demo

After the application is deployed, the IP address can be obtained by the following command:

 $ kubectl get secret outputs-ecs --template={{.data.this_public_ip}} | base64 --decode
["121.196.106.174"]

You can access the management page of the FRP service through the IP:9091 address. The initial account password is: admin:vela123 So far, we have completed the deployment of the ECS service.

Using FRP Services

The use of the FRP client is very simple, we can provide a public IP to any service within the cluster.

1. Deploy FRP-Proxy separately.

 cat <<EOF | vela up -f -
apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
  name: frp-proxy
spec:
  components:
    - name: frp-proxy
      type: worker
      properties:
        image: oamdev/frpc:0.43.0
        env:
          - name: server_addr
            value: "121.196.106.174"
          - name: server_port
            value: "9090"
          - name: local_port
            value: "80"
          - name: connect_name
            value: "velaux-service"
          - name: local_ip
            value: "velaux.vela-system"
          - name: remote_port
            value: "8083"
EOF

In this case, we specify local_ip via velaux.vela-system, which means we are accessing a Kubernetes service named velaux in the namespace vela-system. You can access the service through public IP: 8083.

2. Deploy the agent and normal application together.

 cat <<EOF | vela up -f -
# YAML begins
apiVersion: core.oam.dev/v1beta1
kind: Application
metadata:
  name: composed-app
spec:
  components:
    - name: web-new
      type: webservice
      properties:
        image: oamdev/hello-world:v2
        ports:
          - port: 8000
            expose: true
    - name: frp-web
      type: worker
      properties:
        image: oamdev/frpc:0.43.0
        env:
          - name: server_addr
            value: "121.196.106.174"
          - name: server_port
            value: "9090"
          - name: local_port
            value: "8000"
          - name: connect_name
            value: "composed-app"
          - name: local_ip
            value: "web-new.default"
          - name: remote_port
            value: "8082"
EOF

After the deployment is completed, the service can be accessed through the public network IP: 8082. Another way to play is to define FRP-Proxy as a Trait and directly mount it on the component that needs to expose the service. I hope you can explore the implementation by reading KubeVela's documentation.

clean up the environment

Use the following command to complete the cleanup action of the application created during the test process:

 vela delete composed-app -y
vela delete frp-proxy -y
vela delete ecs-demo -y

Cloud service components are also destroyed when the application is deleted.

So far, we have described how KubeVela completes the unified description and delivery of cloud services and common applications through a specific use case. I hope you have mastered it and tried diversification in your own environment. Through this case, you should also have a general understanding of the final effect of KubeVela combined with Terraform. More about KubeVela's gameplay can be obtained by reading KubeVela's official documentation.

Reference link:

[1] Terraform CLI**

https://www.terraform.io/downloads

[2] Terraform module

https://github.com/wonderflow/terraform-alicloud-ecs-instance

[3] Kubevela control plane installed

https://kubevela.net/docs/install#1-install-velad

[4] here

https://kubevela.net/docs/platform-engineers/components/component-terraform

[5] FRP

https://github.com/fatedier/frp

[6] Document 1

https://kubevela.net/docs/install#1-install-velad

[7] Document 2

https://kubevela.net/docs/reference/addons/terraform

---

You can learn more about KubeVela and the OAM project in the following materials:

• Project repository: github.com/oam-dev/kubevela Welcome to Star/Watch/Fork!
• The official homepage and documentation of the project: kubevela.io, since version 1.1, Chinese and English documents have been provided, and developers are welcome to translate more language documents.
• Project DingTalk Group: 23310022; Slack: CNCF #kubevela Channel
• Join WeChat group: Please add the following maintainer WeChat account to indicate that you have entered the KubeVela user group:

Click here : Check out the official website of the KubeVela project!