EventBridge event bus and EDA architecture analysis

Author: Ken Meng

As one of the 10 strategic technology trends defined by Gartner, Event-Driven Architecture (EDA) has gradually become the mainstream technology architecture. According to Gartner's estimates, 60% of new digital business solutions will use EDA, and EDA will account for half of the technology stacks that business organizations participate in.

Today's more successful companies have recognized that to maximize operational efficiency and customer experience, they must closely integrate business and technology initiatives. Changes in operational events or business situations are the focus of many companies nowadays. These changes can bring practical and useful information to business leaders, and the main purpose of architecture design is to obtain information from customer contacts, transactions, operations, etc. Insight, the two complement each other. Traditional technologies have historically limited the speed at which companies can gain insights from events, such as batch ETL (extract, transform, load) for recording, collecting, and processing such events. Based on the above background, Alibaba Cloud EventBridge came into being.

EventBridge is an event-driven specific landing product, and it is also the best practice of EDA.

What is Event Driven (EDA)

As early as 2018, the Gartner evaluation report listed the Event-Driven Model as one of the top 10 strategic technology trends, and Event-Driven Architecture (EDA) will become the mainstream of future microservices. The report also makes the following assertions:

• By 2022, the software model of event notification will be the solution for more than 60% of new digital commerce;

• By 2022, more than 50% of business organizations will be participating in an ecosystem of event-driven digital business services.

I really like the line George Santayana said in "The Life of Reason" These who fail to learn History are doomed to repeat it. (Those who do not understand history are doomed to repeat it). Let's take history as a guide to see why architecture has evolved to event-driven.

The diagram above is about the architecture evolution timeline. The architecture itself is not good or bad. It is a set of technical decisions that determine all the functional development of subsequent projects (framework, coding specifications, documents, processes...), so I will not talk about the quality of the selection here, but only talk about why it was introduced. Some framework, what problem in software development this framework solves.

• Monolithic architecture: In a single-node service, all modules of a monolithic application are encapsulated and run in a single process, and communication is done through the same stack calls. In this mode, it is very easy to cause unclear structures and relationships, and it is difficult to change and refactor the system. Like an opaque, sticky, brittle, stiff Big Ball of Mud!

• Layered Architecture: In a classic layered architecture, layers are used in a rather discreet way. That is, a layer can only know the data of the layer below it. In subsequent practical applications, it is more of a way that a layer can access any layer below it. The layered architecture solves the problem of logical separation of the monolithic architecture. Each layer can be replaced by equivalents. It is more standardized to distinguish between layers. At the same time, one layer can be used by several different/higher-level layers. Of course, the layer also has obvious shortcomings. The layer cannot encapsulate everything. For example, a field added to the UI may also need to be added to the DB, and the extra layer will seriously damage the system performance.

• MVC architecture: The reason for the MVC architecture is actually very simple. As the complexity of business systems increases, the so-called "full stack engineer" is no longer applicable to most scenarios. In order to reduce the integration complexity of front-end and back-end, the MVC architecture has been promoted. Among them, Model represents business logic; View represents the view layer, such as a small component of the front-end UI; Controller provides coordination between View and Model, such as converting an operation of the user into business logic. In addition, there are many extension architectures, such as Model-View-Presenter, Model-View-Presenter-ViewModel, Resource-Method-Representation, Action-Domain-Responder. I will not go into details. Interested students can search on the wiki.

• EBI architecture: ie Entity, Boundary (interface), Interactor (control). The EBI architecture treats system boundaries as complete connections, not just views, controllers, or interfaces. The entity of EBI represents the actual entity that holds the data and ends the related behavior, very similar to Alibaba Cloud's POP API. EBI is primarily a back-end concept, which is complementary to MVC.

• Onion Architecture: Onion architecture is a low-coupling, high-cohesion architectural model. All applications are built around an independent object model, the inner layer defines the interface, the outer layer implements the interface, the coupling direction is cohesive to the center, and all the code can be compiled and run independently with the infrastructure.

• SOA Architecture: SOA is the abbreviation of Service Orientated Architure, that is, Service Oriented Architecture. Indicates that each function is provided by an independent service, the service defines a clear callable interface, and the orchestration call between services can complete a complete business. In fact, this architecture is also the most mature and the most commonly used architecture pattern in the current architecture.

After covering all the previous architectural trends, let's go back and look at what an EDA architecture is.

EDA Event-Driven Architecture (Event-Driven Architecture) is a system architecture model whose core capability lies in the ability to discover system "events" or important business moments (such as transaction nodes, site visits, etc.) event to take necessary action. This pattern replaces the traditional "request/response" model in which a service has to wait for a reply before moving on to the next task. Event-driven architecture processes are run by events.

The above picture actually explains the model of the EDA architecture very well, but it is not clear enough, so here we compare it with the single architecture to see the differences between them.

In the above comparison diagram, we can actually see the difference between it and the traditional architecture more clearly. In the general traditional architecture, after the order creation operation occurs, a series of operations are actually completed through a system. The event-driven concept is to convert all operations into the concept of "event", and the downstream decides what system to call to complete what kind of operation by capturing an "event".

Let's go back and look at "events". The important part of the comparison just introduced is actually converting operations into some kind of events for distribution. How do we define this event?

Simply put, an event is a significant change in state that is triggered when the user takes a specific action. Take the 4S store selling cars as an example:

• An event when a customer buys a car and its status changes from For Sale to Sold;
• After a successful transaction, the deduction of the amount from the account is an event;
• After clicking Book a Test Drive, it is an event from the time the appointment information is added to the specified user;

Each event may trigger one or more options in response.

Events In fact, the Cloud Native CNCF Foundation hosted the open source CloudEvents project in 2018, which aims to describe events in a unified and standardized format to enhance interoperability between different services, platforms, and systems. Under the project definition, the general event specification is as follows:

Events are mainly composed of Json bodies, which describe the events that occurred through different fields.

To sum up, event-driven actually encapsulates important business moments into "events", and routes events to downstream systems through an EventBus.

Understand the entire processing process of the EDA architecture, but have not solved what this so-called "EventBus" is?

The above figure is the core logical architecture of EventBus. It consists of Event Producer and Event Consumer. The intermediate links are decoupled through Bus. Is it very similar to a traditional MQ architecture? Don't worry, the complex part of this architecture will be explained in the next landing practice part.

Practical thinking on the implementation of EDA architecture

When we start to introduce the landing practice, let's first look at a classic EDA architecture model:

This is a very classic EDA order architecture, which mainly uses EventBridge and FC function computing (if you are not familiar with FaaS, you can use the FC node as a POD node of ECS or Kubernetes), and use events to drive various businesses to collaborate .

Therefore, the central node (EventBridge) of this block actually has three more important capabilities:

1. For Event Capturing: Capable of capturing events;
2. For Routing: The ability to distribute event routing to downstream through event content;
3. For Event Processing: The ability to desensitize or preliminarily filter & screen events.

Usually, it is difficult to realize these three capabilities. For example: Event Capturing may need to be familiar with Dell Boomi, Snaplogic, MuleSoft, Dataflow, Apache Apex, etc. The Routing part may pass RocketMQ, RabbitMQ, ActiveMQ, Apache Kafka, Event Processing Need to know Apache Storm, Apache Flink. Therefore, the logical architecture mentioned earlier is actually very ideal, and these core capabilities need to be included in order to realize the completed EDA event-driven.

In fact, we can also spy on some information from the architecture just now. The EDA architecture actually doesn't seem that simple, so what are its advantages and disadvantages?

The following is a brief list of the advantages of the EDA architecture in practice:

Loose coupling: The event-driven architecture is a highly loosely coupled and highly distributed architecture model. The creator (source) of an event only knows the event that occurred, but does not know how the event is handled, and also cares about how many related parties subscribe to the event;

Asynchronous execution: The EDA architecture is the most suitable execution tool in asynchronous scenarios. We can keep the required events in the queue until the state is normal and executed;

Scalability: The event-driven architecture can quickly divide services through routing & filtering capabilities, providing more convenient expansion and routing distribution;

Agility: Event-driven architectures can provide more agile and efficient deployment scenarios by distributing events anywhere.

Of course, the disadvantages are also obvious:

Complex architecture: The event-driven architecture is complex, with many routing nodes, complex system formation, and many functional requirements;

Difficulty routing distribution: Event routing is difficult to distribute, flexible event routing needs to rely on powerful real-time computing capabilities, and has high requirements for the overall distribution system;

Untraceable: Event tracking is the guarantee of the entire EDA architecture. It is often difficult to track the event processing status in the EDA architecture, requiring a lot of customized development;

Poor reliability: Event-driven is usually less reliable due to the need for multi-system integration, and delivery cannot be guaranteed.

How to Solve the Dilemma in EDA Scenarios

In response to these problems faced by EDA scenarios, Alibaba Cloud has launched EventBridge, a serverless event bus service whose mission is to serve as a hub for cloud events, connect cloud products and applications, applications and applications with the standardized CloudEvents 1.0 protocol, and provide centralized event governance and driving capabilities, helping users to easily build a loosely coupled and distributed event-driven architecture; in addition, there are a large number of SaaS services in vertical fields in the cloud market other than Alibaba Cloud, EventBridge will provide excellent cross-product, cross-organization And cross-cloud integration and integration capabilities help customers create a complete, event-driven, efficient and controllable cloud experience.

Alibaba Cloud defines EventBridge, and its core values include:

• Unified event hub: unify event interface, define event standards, and break the event silos of cloud products;
• Event-driven engine: massive event sources, millisecond-level triggering capability, accelerates EDA/Serverless architecture upgrades;
• Openness and integration: Provides rich cross-product and cross-platform connectivity capabilities to facilitate the integration of cloud products, applications, and SaaS services.

The following introduces EventBridge from the architectural and functional levels:

architectural level

For complex architecture problems, EventBridge provides the industry's common Source, Buses, Rules, Targets module management capabilities, and supports both EventBus and EventStream modes, which greatly reduces the difficulty of event-driven architecture.

1) Event bus model The N:N model of the classic EDA (event-driven) scenario provides core capabilities such as multi-event routing, event matching, and event conversion, helping developers to quickly build an event-driven architecture.

2) Event Streaming Model Standard Streaming (1:1) Streaming processing scenario, no bus concept, used for end-to-end data dump, data synchronization and data processing, etc., to help easily build end-to-end data pipeline services on the cloud.

functional level

At the functional level, the core highlight applications of EventBridge include:

1) Event rule-driven <br>For event-based routing distribution, EventBridge is driven by event rules, supports 8 major event modes, and 4-fold converters to meet all the demands of routing distribution.

2) Event Tracking <br>For events that cannot be tracked, it exclusively provides event tracking capabilities and event analysis/query capabilities. Perfect full-link event query and analysis capabilities for users.

3) DLQ/retry mechanism, full-process triggering of events <br>For poor reliability, it supports DLQ/retrying mechanism and triggering of full-process events, which greatly guarantees event failures and delays caused by users' downstream systems.

4) Schema registration center <br>For complex event management, it supports Schema registration center, supports event information interpretation, preview and upstream and downstream code generation capabilities, and helps users to complete the sending and receiving of events with low code. Solve a series of incident management problems such as cross-departmental information communication difficulties and business code redundancy.

5) At the same time, based on the above functions, EventBridge supports docking with more than 85 Alibaba Cloud products and 847 event types.

For more product function introductions, please visit the EventBridge official website

https://www.aliyun.com/product/aliware/eventbridge

Alibaba Cloud EventBridge More Scenarios

Classic EDA Event Driven

The most important capability of EventBridge is to build EDA (Event-driven Architectures) event-driven architecture by connecting applications, cloud services and serverless services, driving the connection between applications and applications, and applications and clouds.

Streaming ETL scenarios

Another core capability of EventBridge is to provide basic filtering and transformation capabilities for the responsibility of streaming data pipelines, data synchronization/crossing between different data warehouses, data processing programs, data analysis and processing systems In scenarios such as regional backup, connect different systems and different services.

Unified Event Notification Service

EventBridge provides a wealth of cloud product event sources and event lifecycle management tools. You can directly monitor the data generated by cloud products through the bus and report it to downstream services such as monitoring and notification.

Heavy recommendation

This article is an overall introduction to the EventBridge event bus and EDA architecture. If you are still in the mood and want to know more scenario applications, you can pay attention to "Alibaba Cloud EventBridge Series Open Courses", the complete course is now available! This series of live courses includes 5 topics in total, which will take you to understand the core functions and applications of Alibaba Cloud EventBridge event bus.

The specific arrangement of the later series of courses is as follows, interested friends don't miss it~

Click here to watch the public class video corresponding to this article~