Quickly deploy MQTT clusters on AWS using Terraform

Terraform is an infrastructure-as-code (IaC) tool launched by HashiCorp, which includes low-level components such as computing instances, storage, and networking, as well as high-level components such as DNS, LBS, and so on. Users can use Terraform to safely and efficiently build, change and update infrastructure.

In the traditional private cloud or public cloud deployment method, users need to deploy the infrastructure (virtual machine, network, storage, etc.) first, and then start to deploy the MQTT cluster. With Terraform, users can do both at the same time. In addition, the same set of tools can be deployed on different platforms, and resources can be defined and configured in a repeatable and predictable way through templates, which can greatly reduce errors caused by human factors.

This article will take the distributed IoT MQTT message server EMQX as an example, and use AWS as the public cloud platform to introduce how to use Terraform to quickly deploy a highly available MQTT cluster .

Introduction to Terraform

As a tool for managing the lifecycle of services, Terraform can record and track all environmental changes with state files. The default state is stored locally and is defined by HCL or JSON, which is the template language provided by HashiCorp.

• Coding: Use HCL to write infrastructure code. Blocks, parameters and expressions can be defined.
• Plan: Run Terraform Plan to check that the execution plan meets expectations.
• Apply: Run Terraform Apply to build the infrastructure your users need

Installation and preparation

Install Terraform

Take Mac as an example, install it through brew.

 brew tap hashicorp/tap 2brew install hashicorp/tap/terraform

Verify installation

 terraform -help

Refer to the official documentation for details

Install AWS Cli

Take Mac as an example

 curl "https://awscli.amazonaws.com/AWSCLIV2.pkg" -o "AWSCLIV2.pkg"
sudo installer -pkg AWSCLIV2.pkg -target /

Verify installation

 which aws
aws --version

Refer to official documentation

Add user

1. Go to the AWS IAM menu and click "Add User"

   

2. In adding users, remember to check "Access Key"

   

3. Select "Create Group"

   

4. Add the "AdministratorAccess" policy in the creation group

   

5. Click "Create User" to complete the user addition

   

6. Get Access Key and Secret

   

Configure AWS Access Key

Get the access key and secret created above, and set the environment variables

 AWS_ACCESS_KEY_ID: ${anaccesskey}
AWS_SECRET_ACCESS_KEY: ${asecretkey}

Deploy an EMQX cluster on AWS using Terraform

Download the AWS deployment script

 git clone "https://github.com/emqx/terraform-emqx-emqx-aws.git"

Deployment script description:

• EMQX 5.X is not supported temporarily
• AWS CLI: aws-cli/2.2.41 Python/3.8.8 Darwin/21.4.0 exe/x86_64 prompt/off

Script configuration file path:

• Single-machine deployment configuration file: services/emqx/terraform.tfvars
• Cluster deployment configuration file: services/emqx_cluster/terraform.tfvars

The deployment script uses the following configuration by default, and readers can modify the terraform.tfvars file according to the actual situation:

• Default EMQX version: Enterprise 4.4.3

If you want to deploy the open source version, you need to modify the emqx_package value, such as deploying the open source version 4.4.3: https://www.emqx.com/en/downloads/broker/v4.4.3/emqx-4.4.3-otp23.3.4.9-3-ubuntu20.04-amd64.zip

• The default AWS Region is us-east-1
• Default cluster nodes: 3 t3.small

Deploy EMQX cluster

Users can change the default configuration by modifying the terraform.tfvars file

 cd services/emqx_cluster
terraform init
terraform plan
terraform apply -auto-approve

Wait for a few minutes, the result of the deployment is as follows

Verify EMQX Cluster Deployment Results

After the cluster is successfully deployed, we can simply test whether the cluster is running normally. First obtain the IP address of the cluster from the above figure, and use the following information to access the Dashboard of EMQX Enterprise Edition.

 http://tf-elb-nlb-5bff6976b15586dd.elb.us-east-1.amazonaws.com:18083

Username: admin

Password: public

As can be seen from the above figure, we have deployed 3 nodes, which can be easily verified through the Websocket tool:

1. On the left menu bar, select Tools→WebSocket

   

2. Click "Connect"

   

3. Subscribe to topics

   

4. Click Publish, you can see the following results normally

   

Epilogue

So far, we have completed the entire process of rapidly deploying an MQTT cluster on AWS through Terraform. Readers can modify the deployment script according to the actual situation, create an EMQX cluster that meets their own business needs, and build a competitive IoT platform and application with the help of EMQX's advantages in IoT data connection, mobility, and processing.

refer to:

Code repository: https://github.com/emqx/terraform-emqx-emqx-aws

EMQX Terraform module official documentation: https://docs.emqx.com/zh/emqx-terraform/latest

Copyright statement: This article is original by EMQ, please indicate the source when reprinting.

Original link: https://www.emqx.com/zh/blog/deploy-mqtt-cluster-on-aws-using-terraform