Introduction to EventBridge Features | Using EventBridge in an IaC Way

Author: Wang Chuan (Fording)

introduction

As the infrastructure for building EDA architecture, EventBridge provides flexible and rich event collection, processing and routing capabilities through some core concepts and features. For many users, using EventBridge through a convenient guide in the console should be the fastest way to get started. In addition, there are also many users who are faced with the management of a large number of cloud products, and the way of using the console to manage each resource has become a heavy manual operation burden.

In order to solve this problem, it is now possible to bring the capabilities of EventBridge to users conveniently and quickly through OpenAPI, terraform, etc. This article will focus on key concepts and features of EventBridge and IaC, and then demonstrate how to apply IaC concepts to automate the deployment of EventBridge to use these concepts and features.

EventBridge overview

event-driven architecture

Event-driven architecture is a loosely coupled and distributed drive architecture. After collecting events generated by an application, necessary processing is performed on the events in real time, and then the events are routed to the downstream system without waiting for the system to respond. Using the event bus EventBridge, various simple or complex event-driven architectures can be built to connect cloud products and applications, applications and applications, etc. with the standardized CloudEvents 1.0 protocol.

The event-driven architecture architecture has the following three capabilities:
• Event collection : responsible for collecting events that occur in various applications, such as new orders, return orders and other status changes;
• Event processing : desensitize events, and conduct preliminary filtering and screening of events;
• Event routing : Analyze event content and distribute event routing to downstream products.

Event-driven architecture has the following advantages:

• Reduce coupling : reduce the coupling between event producers and subscribers. The event producer only needs to pay attention to the occurrence of the event, and does not need to pay attention to how the event is processed and which subscribers it is distributed to; the failure of any link will not affect the normal operation of other businesses;
• Asynchronous execution : The event-driven architecture is suitable for asynchronous scenarios. Even during peak demand periods, events from various sources are collected and retained in the event bus, and then gradually distributed and delivered, without causing system congestion or excess resources;
• Scalability: The routing and filtering capabilities in the event-driven architecture support the division of services, facilitating expansion and routing distribution;
• Agility: The event-driven architecture supports integration with various Alibaba Cloud products and applications, supports event routing to any system service, and provides various agile and efficient deployment solutions.

Building an EDA Architecture with EventBridge

Event bus EventBridge is a serverless event bus service provided by Alibaba Cloud. Several core concepts provided by EventBridge can meet the needs of building EDA architecture.

The event bus EventBridge supports the following event sources:

**• Alibaba Cloud Official Event Source• Custom Event Source**

Event Bus EventBridge's event bus includes the following types:

• Dedicated event bus for cloud services : a built-in event bus that does not need to be created and cannot be modified, used to receive events from your Alibaba Cloud official event source; events from Alibaba Cloud official event sources can only be published to the dedicated cloud service bus;
• Custom event bus : You need to create and manage an event bus to receive events of custom applications or stock message data; events of custom applications or stock message data can only be published to the custom bus.

In EventBridge, an event rule contains the following:
• Event mode : used to filter events and route events to event targets;
• Event target : including event conversion and processing, responsible for consuming events.

EventBridge provides a concise event pattern matching syntax and flexible event conversion capabilities. Some specific examples will be shown later through demonstrations.

In addition, EventBridge provides several enhancements that make the events flowing through the EDA architecture more transparent, enabling out-of-the-box observation and analysis:

• Event Tracking : You can view the event content and processing track published to the event bus EventBridge;
• Event Analysis : Query, analyze, process and visualize various events published to the event bus, so as to discover the intrinsic value of events.

Introduction to IaC

After introducing the basic content of the event bus EventBridge, let's learn about IaC together. In the practice of DevOps, IaC is a very important part. By coding and versioning the infrastructure, it is easy to use version control tools to provide single source of truth, coordinate changes with multi-person cooperation, implement strict reviews, Use some CI/CD pipeline tools (even GitOps) to trigger deployments automatically. Software system developers only need to make a small effort to describe the requirements, and then they can get the required virtual machine, network and other cloud services in a few minutes, which greatly shortens the deployment time, and can also ensure multiple environments. The configuration consistency also reduces the probability of introducing errors by reducing human manipulation.

There are generally two ways in IaC code practice, imperative and declarative.

• Imperative : As the name implies, it is necessary to explicitly issue the instructions for each action, and the description is How, such as "create an ECS of xx specification". The code needs to carefully arrange the sequence of each step, deal with various possible errors, and especially pay attention to handling the impact of each change on the existing resources, otherwise a little carelessness may cause service interruption. For example, as a developer, you can use your familiar programming language to call Alibaba Cloud's OpenAPI to manage resources, because these APIs are similar to Create, Describe, Delete and other operations, which is an imperative IaC practice.

• Declarative: It means that the developer only describes what the final state of his requirements is, that is, what is described, such as "an ECS of xx specification". Students who are familiar with Kubernetes should be familiar with this concept. The IaC tool can automatically arrange the order by describing the dependencies between resources. If there are existing resources, it compares the difference between the expected state and the actual state, and makes an update according to the difference; if it does not exist, it needs to be created. It can be seen that declarative is very friendly to developers and greatly reduces the mental burden of developers.

Advantages of IaC:
• Reduce costs: manage resources efficiently and reduce the amount of manpower to do so;
• Improve efficiency : speed up resource delivery and software deployment;
• Risk Control :
• reduce errors;
• Improve infrastructure consistency;
• Eliminate configuration drift

As a leader in IaC, terraform provides a powerful ability to automate infrastructure management. The ecosystem is rich, and many cloud vendors provide official plug-ins. Most of Alibaba Cloud's products (including EventBridge) fully support terraform, making it extremely easy to deploy infrastructure across multiple clouds. Since it is IaC, terraform provides its own language HCL (hashicorp configuration language). HCL has a concise syntax similar to json. Through declarative resource description, developers can get started quickly.

Hands

Ready to work

• Install the terraform cli tool, which can be found at https://www.terraform.io/cli .
• Create a tf file terraform.tf with the following content (replace the values in <>)

 provider "alicloud" {
  access_key = "<your access key>"
  secret_key = "<your secret key>"
  region = "<region id>"
}

Case 1: Monitoring resource changes on the cloud through DingTalk

Suppose a user uses a lot of cloud resources as a production environment and needs to perceive changes in online resources. A feasible solution is to use EventBridge to deliver audit events from ActionTrail to the user's DingTalk.
First create a robot according to DingTalk's official documentation, write down the webhook url and the signed secret key, which will be used later.
Create a tf file 1_actiontrail2dingding.tf with the following contents (the values in <> need to be replaced)

 # 案例1：通过钉钉监控云上资源变化
# 目标：
# - 熟悉部署使用EventBridge的default总线
# - 熟悉EventBridge的事件模式匹配
# - 熟悉EventBridge的事件转换配置
# 声明一个default总线上的规则
resource "alicloud_event_bridge_rule" "audit_notify" {
  # default总线默认存在，所以这里可以直接使用
  event_bus_name = "default"
  rule_name      = "audit_notify"
  description    = "demo"
  # 通过后缀匹配的方式过滤来自所有云产品事件源的ActionTrail:ApiCall事件
  # 其他更多模式匹配的介绍可以查阅文档：https://help.aliyun.com/document_detail/181432.html
  filter_pattern = jsonencode(
    {
      "type" : [
        {
          "suffix" : ":ActionTrail:ApiCall"
        }
      ]
    }
  )
  targets {
    target_id = "test-target"
    endpoint  = "<your dingtalk bot webhook url>"
    # type的取值可以查阅文档：https://registry.terraform.io/providers/aliyun/alicloud/latest/docs/resources/event_bridge_rule#type
    type      = "acs.dingtalk"
    # 每个事件目标都有一组对应的param_list，具体可以查阅文档：https://help.aliyun.com/document_detail/185887.html
    # 每一个param的form关系到事件转换的配置，可以查阅文档：https://help.aliyun.com/document_detail/181429.html
    param_list {
      resource_key = "URL"
      form         = "CONSTANT"
      value        = "<your dingtalk bot webhook url>"
    }
    param_list {
      resource_key = "SecretKey"
      form         = "CONSTANT"
      value        = "<your dingtalk bot secret key>"
    }
    # 这里展示了TEMPLATE类型的事件转换描述
    # value是使用jsonpath引用事件内容的字典，template则是模板内容，EventBridge最终会根据这两者结合事件本身渲染出这个参数的值
    param_list {
      resource_key = "Body"
      form         = "TEMPLATE"
      value        = jsonencode(
        {
          "source": "$.source",
          "type": "$.type"
          "region": "$.data.acsRegion",
          "accountId" : "$.data.userIdentity.accountId",
          "eventName" : "$.data.eventName",
        }
      )
      template = jsonencode(
        {
          "msgtype" : "text",
          "text" : {
            "content": "来自 $${source} 的 $${type} 审计事件：$${accountId} 在 $${region} 执行了 $${eventName} 操作"
          }
        }
      )
    }
  }
}

Execute the commands one by one in the command line window:

• Initialize terraform init
• Preview changes to terraform plan
• Apply changes terraform apply

Operate in the cloud product console, here KMS is used as an example

Dingding receives a message notification

View event traces in the EventBridge console

Case 2: Custom bus triggers FunctionCompute

Suppose a user's application generates some events, and one of the links is to flexibly process these events through FunctionCompute. Then you can implement this solution through EventBridge's custom event source and function calculation event target.

Create a python script file src/index.py that simulates processing events, with the following contents:

 # -*- coding: utf-8 -*-
import logging
def handler(event, context):
  logger = logging.getLogger()
  logger.info('evt: ' + str(event))
  return str(event)

Create a tf file 2_trigger_function.tf with the following content (the values in <> need to be replaced)

 # 案例2：自定义总线触发FunctionCompute
# 目标：
# - 熟悉部署使用EventBridge的自定义总线
# - 熟悉"自定义应用"事件源配置
# - 熟悉“FunctionCompute”事件目标配置
# 由于用户自己产生的事件需要投递到自定义总线，这里声明一个叫demo_event_bus的自定义总线
resource "alicloud_event_bridge_event_bus" "demo_event_bus" {
  event_bus_name = "demo_event_bus"
  description    = "demo"
}
# 声明一个在demo_event_bus总线上的自定义事件源，用于通过sdk或者控制台向EventBridge投递事件
resource "alicloud_event_bridge_event_source" "demo_event_source" {
  event_bus_name         = alicloud_event_bridge_event_bus.demo_event_bus.event_bus_name
  event_source_name      = "demo_event_source"
  description            = "demo"
  linked_external_source = false
}
# 声明一个叫fc_service的函数计算服务，publish=true意味着会立即部署上传的函数代码。
resource "alicloud_fc_service" "fc_service" {
  name        = "eb-fc-service"
  description = "demo"
  publish     = true
}
# 将前面准备的python脚本文件打包成zip用于部署到函数计算
data "archive_file" "code" {
  type        = "zip"
  source_file = "${path.module}/src/index.py"
  output_path = "${path.module}/code.zip"
}
# 声明一个fc_service服务中的函数，其中filename引用了上面描述的zip包，会将这个代码包上传。
resource "alicloud_fc_function" "fc_function" {
  service     = alicloud_fc_service.fc_service.name
  name        = "eb-fc-function"
  description = "demo"
  filename    = data.archive_file.code.output_path
  memory_size = "128"
  runtime     = "python3"
  handler     = "index.handler"
}
# 声明一个在demo_event_bus总线上的规则
resource "alicloud_event_bridge_rule" "demo_rule" {
  event_bus_name = alicloud_event_bridge_event_bus.demo_event_bus.event_bus_name
  rule_name      = "demo_rule"
  description    = "demo"
  # 通过匹配source过滤来自于前面创建的自定义事件源的事件
  filter_pattern = jsonencode(
    {
      "source" : ["${alicloud_event_bridge_event_source.demo_event_source.id}"]
    }
  )
  targets {
    target_id = "demo-fc-target"
    # type的取值可以查阅文档：https://registry.terraform.io/providers/aliyun/alicloud/latest/docs/resources/event_bridge_rule#type
    type      = "acs.fc.function"
    endpoint  = "acs:fc:<region id>:<your account id>:services/${alicloud_fc_service.fc_service.name}.LATEST/functions/${alicloud_fc_function.fc_function.name}"
    param_list {
      resource_key = "serviceName"
      form         = "CONSTANT"
      value        = alicloud_fc_service.fc_service.name
    }
    param_list {
      resource_key = "functionName"
      form         = "CONSTANT"
      value        = alicloud_fc_function.fc_function.name
    }
    param_list {
      resource_key = "Qualifier"
      form         = "CONSTANT"
      value        = "LATEST"
    }
    # 注意form=ORIGINAL意味着每次投递事件都会将事件的原始内容作为这个参数的值
    param_list {
      resource_key = "Body"
      form         = "ORIGINAL"
    }
  }
}

Execute commands sequentially in the command line window

• Initialize terraform init
• Preview changes to terraform plan
• Apply changes terraform apply

Simulate a custom event source to publish events in the console

View function call logs on the console page of FunctionCompute

View event traces in the EventBridge console

Summarize

As the infrastructure for building EDA architecture, EventBridge provides flexible and rich event collection, processing and routing capabilities through some core concepts and features, and supports the convenient and quick delivery of these capabilities to users through OpenAPI, terraform, etc. This article introduces the key concepts and features of EventBridge and IaC, and then demonstrates how to apply the IaC philosophy to automate the deployment of EventBridge to use these concepts and features.

We look forward to discovering more ideas that use EventBridge to quickly build EDA architecture, and use terraform to quickly turn these ideas into reality.

Related Links

[1] Alibaba Cloud terraform documentation
https://help.aliyun.com/product/95817.html

[2] terraform registry documentation
https://registry.terraform.io/providers/aliyun/alicloud/latest/docs/resources/event_bridge_event_bus

[3] DingTalk official documentation
https://open.dingtalk.com/document/group/custom-robot-access

To learn more about EventBridge, scan the QR code below to join the DingTalk group~

Click here to watch the video corresponding to the article~