Terraform preliminary exploration: migrating local projects to Terraform Cloud for execution

Previous article We tried using Terraform in the local environment to create and manage AWS Lightsail resources. For managing some cloud resources, we need to install the corresponding CLI tools locally and configure the credentials to access the corresponding cloud resources (such as AWS CLI, AccessKeyID etc.), Terraform by calling the local CLI tools or cloud API to manage cloud resources of the state, which is used by default local type of Backend, the state of the resource file ( .tfstate ) is stored in the local file directory.
In this article, we will try to use remote type Backend to migrate the project to Terraform Cloud for execution, and Terraform Cloud will manage the resource status.

What is Terraform Cloud

Terraform Cloud is a SaaS application that manages Terraform running in a consistent and reliable environment, which can replace the execution of Terraform projects on the local machine. It stores shared state and confidential data, and can be connected to a version control system (such as Git). Allows us to work on the infrastructure as code in the team.
Terraform is a commercial application, team and commercial use will charge fees and provide more advanced features. But for individual users, basic functions can be used for free. For details of fees and functions, please refer to ( https://www.hashicorp.com/products/terraform/pricing).

First, we need to register a Terraform Cloud account, visit https://app.terraform.io/signup/account , and follow the prompts to register for a free account

When you log in to Terraform Cloud for the first time after registration, it will ask how to start a project. Here we choose Start from scratch , which means we will start with an empty template

Next we need to create an organization (Organization). For example, here I create an learn-terraform . An organization is similar to giving a namespace, which can manage multiple workspaces and under Variables and environment variables shared by the workspace.

Next, we need to log in to Terraform Cloud in the local environment and add the corresponding configuration to reinitialize the project.

Re-initialize the project

After completing the account registration of Terraform Cloud, we need to run terraform login on the local terminal, open the browser to log in to the account to get a Token value, copy it into the terminal to complete the login

> terrafrom login
Terraform must now open a web browser to the tokens page for app.terraform.io.

If a browser does not open this automatically, open the following URL to proceed:
    https://app.terraform.io/app/settings/tokens?source=terraform-login


---------------------------------------------------------------------------------

Generate a token using your browser, and copy-paste it into this prompt.

Terraform will store the token in plain text in the following file
for use by subsequent commands:
    /home/mengz/.terraform.d/credentials.tfrc.json

Token for app.terraform.io:
  Enter a value:

Then we modify the project configuration file main.tf and add backend "remote"

terraform {
  backend "remtoe" {
    organization = "learn-terraform"
    workspaces {
      name = "mylightsail"
    }
  }

  ...
}

Execute terraform init , Terraform will download the remote plug-in, connect to Terraform Cloud's learn-terraform/mylightsail workspace, and migrate the local state files to the cloud

$ terraform init

Initializing the backend...Do you want to copy existing state to the new backend?  Pre-existing state was found while migrating the previous "local" backend to the  newly configured "remote" backend. No existing state was found in the newly  configured "remote" backend. Do you want to copy this state to the new "remote"  backend? Enter "yes" to copy and "no" to start with an empty state.
  Enter a value: yes
Releasing state lock. This may take a few moments...
Successfully configured the backend "remote"! Terraform will automaticallyuse this backend unless the backend configuration changes....

The browser accesses the Terraform Cloud WebUI and enters the corresponding workspace to view the status information.

After completion, you can delete the .terraform/terraform.tfstate The local project has used Terraform Cloud as a remote backend and is associated with a command line (CLI) driver. Therefore, you can update the resource configuration file locally and then run the plan & apply command locally, which will trigger the remote backend. Perform specific state maintenance work on the end Cloud. But to use Terraform Cloud to perform state maintenance, we also need to configure AWS access credentials to Terraform Cloud.

Configure environment variables for the workspace

To use Terraform Cloud to maintain cloud resources (such as AWS), we need to configure the corresponding access credentials. Here we need to configure AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY AWS as environment variables of the project space.
Variables tab in the workspace, and click the + Add Varaible button

Select Environment Variables , then add AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY , and set the corresponding values.

After completion, we can run terraform plan on the local console, terraform apply send the operation to Terraform Cloud to run. Of course, we can still execute terraform show in the local project to view the current status, and the status will be managed in the cloud

> terraform plan
Running plan in the remote backend. Output will stream here. Pressing Ctrl-C
will stop streaming the logs, but will not stop the plan running remotely.

Preparing the remote plan...

To view this run in a browser, visit:
https://app.terraform.io/app/mengz-infra/my-lightsail/runs/run-LzwFBbihffEKmucd

Waiting for the plan to start...

Terraform v1.0.11
on linux_amd64
Configuring remote state backend...
Initializing Terraform configuration...
aws_lightsail_static_ip.outline-sig-ip: Refreshing state... [id=Outline-EIP]
aws_lightsail_instance.outline-sig: Refreshing state... [id=Outline-Sig]
aws_lightsail_instance_public_ports.outline-sig-public-ports: Refreshing state... [id=Outline-Sig-987241840]
aws_lightsail_static_ip_attachment.outline-sig-ip-attache: Refreshing state... [id=Outline-EIP]

No changes. Your infrastructure matches the configuration.

Terraform has compared your real infrastructure against your configuration
and found no differences, so no changes are needed.

You can see that the plan is running on the remote backend.

Version management project

Finally, we can submit the configuration file of the project to the version control system (such as Gitlab), and configure the version control of the workspace

In the settings of the Terraform Cloud workspace, follow the prompts to configure the associated version management code repository. After completion, after we submit the updated code locally, it will automatically trigger Terraform Cloud to execute and maintain the new state. But this will not allow execution of terraform apply

> terraform apply

│ Error: Apply not allowed for workspaces with a VCS connection
│ A workspace that is connected to a VCS requires the VCS-driven workflow to ensure that the VCS remains the single source of truth.

State maintenance can only be triggered by updating the code and submitting it to the remote code warehouse. This will make it easier to share infrastructure code with the team, and to maintain the state of the infrastructure together, and it will also become more of the way GitOps works.

Summarize

Based on an article - try to use Terraform Lightsail resources in local environmental management AWS, extended to apply to the state management of migration operation to try to Terraform Cloud as a remote back-end, in addition to Terraform Cloud after, there are other types of Backend , You can refer to ( https://www.terraform.io/docs/language/settings/backends/index.html).
Since then, we have explored the use of Terraform as an IaC tool to manage AWS Lightsail resources as an introduction to Terraform learning. Hashicrop official offers more learning resources and documentation , want in-depth study Terrform, and put into practice, also refer to the official document .

[At the same time published at Terraform Preliminary: Migrating local projects to Terraform Cloud to execute ]