In-depth practice of state machine engine in vivo marketing automation | Engine 02

1. Business Background

The marketing automation platform supports a variety of different types of operational activity strategies (such as: SMS push strategy, WeChat image and text push strategy, App Push strategy), and each type of activity has its own different execution process and activity status. For example, the activity execution process of the SMS activity is as follows:

(Figure 1-1: SMS activity status transition)

The entire SMS campaign has gone through multiple status change processes: not started → data preparation → data ready → activity push → activity ended. Not only that, we found that in the process of active business logic processing, there are the following similar characteristics:

• Each time a new activity business type is added, the corresponding activity state and the logic for processing state changes must be added;
• When a type of activity business process is modified, it may be necessary to make changes to the original state transfer process;
• When each business writes its own business code for state transfer, the coupling between the core business logic and the control logic will be very strong, the scalability will be poor, and the cost will be high.

For the flow management of system state, there is a set of standard theoretical solution model in computer field - finite state machine.

Second, understand the state machine

2.1 State Machine Definition

Finite-State Machine (Finite-State Machine, abbreviation: FSM), referred to as state machine. is a model that represents a finite number of states and transitions and trigger actions between these states.

• A state describes the state of a system object at a certain moment.
• Transitions indicate state changes, and are generally conditional triggering of state transitions through external events.
• Actions are actions to be performed on a given state.

In short, a state machine is composed of events, states, and actions. The relationship between the three is: an event triggers the transition of the state, and the transition of the state triggers the execution of subsequent actions. Actions are not required, and only state transitions can be performed without any operations.

(Figure 2-1: State Machine Composition)

Therefore, using the state machine model to describe the above [Figure 1-1: SMS activity state transition] is:

(Figure 2-2: SMS activity state machine)

A state machine is essentially a mathematical modeling of a system that systematically expresses the solution to a problem. Let's take a look at the ways in which state machines are implemented in actual development.

2.2 Implementation of the state machine

2.2.1 Implementation of Condition-Based Judgment

This is the most direct way of implementation. The so-called conditional judgment is implemented by hard-coding using if-else or switch-case branch judgment. For the previous SMS activity, the code example based on the condition judgment method is as follows:

 /**
  * 短信活动状态枚举
  */
public enum ActivityState {
    NOT_START(0), //活动未开始
    DATA_PREPARING(1), //数据准备中
    DATA_PREPARED(2), //数据已就绪
    DATA_PUSHING(3), //活动推送中
    FINISHED(4); //活动结束
}
 
/**
  * 短信活动状态机
  */
public class ActivityStateMachine {
    //活动状态
    private ActivityState currentState;
    public ActivityStateMachine() {
        this.currentState = ActivityState.NOT_START;
    }
    /**
     * 活动时间开始
     */
    public void begin() {
        if (currentState.equals(ActivityState.NOT_START)) {
            this.currentState = ActivityState.DATA_PREPARING;
            //发送通知给运营人员
            notice();
        }
        // do nothing or throw exception ...
    }
 
    /**
     * 数据计算完成
     */
    public void finishCalData() {
        if (currentState.equals(ActivityState.DATA_PREPARING)) {
            this.currentState = ActivityState.DATA_PREPARED;
            //发送通知给运营人员
            notice();
        }
        // do nothing or throw exception ...
    }
 
     /**
     * 活动推送开始
     */
    public void beginPushData() {
        //省略
    }
     /**
     * 数据推送完成
     */
    public void finishPushData() {
        //省略
    }
}

The state transition and action triggering are controlled by conditional branch judgment. The above if judgment condition can also be replaced with a switch statement, and the current state is used as a branch to control the operations that can be performed in this state.

Applicable scene

It is suitable for scenarios where the number of business states is small or the logic of jumping between states is relatively simple.

defect

When the corresponding relationship between the trigger event and the business state is not simple one-to-one, multiple conditional branch judgments need to be nested, and the branch logic will become extremely complex; when the state process changes, the branch logic needs to be changed, which does not meet the Open-closed principle, code readability and scalability are very poor.

2.2.2 Implementation based on state pattern

If you understand design patterns, you can easily associate the two concepts of state machine and state pattern. In fact, state pattern can be used as an implementation of state machine. The main implementation idea is to separate the behavior of different states through the state mode, and call different methods corresponding to different states according to the changes of state variables. The code example is as follows:

 /**
   * 活动状态接口
   */
interface IActivityState {
    ActivityState getName();
    //触发事件
    void begin();
    void finishCalData();
    void beginPushData();
    void finishPushData();
}
 
 /**
   * 具体状态类—活动未开始状态
   */
public class ActivityNotStartState implements IActivityState {
    private ActivityStateMachine stateMachine;
    public ActivityNotStartState(ActivityStateMachine stateMachine) {
        this.stateMachine = stateMachine;
    }
 
    @Override
    public ActivityState getName() {
        return ActivityState.NOT_START;
    }
 
    @Override
    public void begin() {
        stateMachine.setCurrentState(new ActivityDataPreparingState(stateMachine));
        //发送通知
        notice();
    }
 
    @Override
    public void finishCalData() {
        // do nothing or throw exception ...
    }
    @Override
    public void beginPushData() {
        // do nothing or throw exception ...
    }
    @Override
    public void finishPushData() {
        // do nothing or throw exception ...
    }
}
 
 /**
   * 具体状态类—数据准备中状态
   */
public class ActivityDataPreparingState implements IActivityState {
    private ActivityStateMachine stateMachine;
    public ActivityNotStartState(ActivityStateMachine stateMachine) {
        this.stateMachine = stateMachine;
    }
 
    @Override
    public ActivityState getName() {
        return ActivityState.DATA_PREPARING;
    }
    @Override
    public void begin() {
        // do nothing or throw exception ...
    }
    public void finishCalData() {
        stateMachine.setCurrentState(new ActivityDataPreparedState(stateMachine));
        //TODO:发送通知
    }
   @Override
    public void beginPushData() {
        // do nothing or throw exception ...
    }
    @Override
    public void finishPushData() {
        // do nothing or throw exception ...
    }
}
    ...(篇幅原因，省略其他具体活动类)
 
 
 /**
   * 状态机
   */
public class ActivityStateMachine {
    private IActivityState currentState;
    public ActivityStateMachine(IActivityState currentState) {
        this.currentState = new ActivityNotStartState(this);
    }
    public void setCurrentState(IActivityState currentState) {
        this.currentState = currentState;
    }
    public void begin() {
        currentState.begin();
    }
    public void finishCalData() {
        currentState.finishCalData();
    }
    public void beginPushData() {
        currentState.beginPushData();
    }
    public void finishPushData() {
        currentState.finishCalData();
    }
}

The state pattern defines the state-behavior correspondence, and encapsulates the behavior of each state in the corresponding state class. We only need to extend or modify the specific state class to achieve the requirements of the corresponding process state.

Applicable scene

It is suitable for scenarios with few business states and simple state transitions. Compared with the previous if/switch conditional branching method, when the business state process is added or modified, the impact granularity is smaller, the scope is controllable, and the scalability is stronger.

defect

It is also difficult to deal with complex business process state transition scenarios. In this scenario, using the state pattern will introduce a lot of state classes and methods. When the state logic changes, the code will also become difficult to maintain.

It can be seen that although the above two methods can realize the triggering, transition and action flow of the state machine, the reusability is very low. It is impossible to build an abstract state machine component that can satisfy most business scenarios.

2.2.3 DSL-based implementation

2.2.3.1 Introduction to DSL

The full name of DSL is Domain-Specific Languages, which refers to a computer programming language with limited expressiveness for a specific field. Different from general-purpose programming languages, DSLs are only used in some specific fields, focusing on solving a certain problem of the system in that field. DSL is usually divided into internal DSL (Internal DSLs), external DSL (external DSLs).

• Internal DSL: System-based host language, DSL written and processed by the host language, such as: Java-based internal DSL, C++-based internal DSL, Javascript-based internal DSL.
• External DSL : Different from the system host language, a DSL written and processed by a custom language or other programming language has an independent parser. For example: regular expressions, XML, SQL, HTML, etc.

(Read more about DSLs: Martin Fowler's "Domain Specific Languages").

2.2.3.2 DSL Selection and State Machine Implementation

Using a DSL as a development tool allows the behavior of a system to be described in a clearer and more expressive form. DSL is also the recommended way to implement state machine at present. You can choose internal DSL or external DSL to implement according to your own needs.

• Internal DSL : If the business system only wants to configure the state machine directly through code, you can choose to use the internal DSL, which is simple and direct, and does not need to rely on additional parsers and components.

Java internal DSL generally uses Builder Pattern and Fluent Interface (Builder Pattern and Streaming Interface) to implement examples:

 StateMachineBuilder builder = new StateMachineBuilder();
                     builder.sourceState(States.STATE1)
                            .targetState(States.STATE2)
                            .event(Events.EVENT1)
                            .action(action1());

• External DSL : It can utilize the parsing capabilities of external storage and common scripting languages to realize dynamic configuration at runtime, support visual configuration and cross-language application scenarios.

An external DSL essentially describes the state transition process in other external languages, such as using XML:

 <state id= "STATE1">
  <transition event="EVENT1"  target="STATE2">
    <action method="action1()"/>
  </transition>
</state>
 
<state id= "STATE2">
</state>

External DSLs are generally placed in external storage such as configuration files or databases. Through the corresponding text parser, the configuration of the external DSL can be parsed into a model similar to the internal DSL for process processing; at the same time, due to the independence and persistence of external storage , which can easily support dynamic changes and visual configuration at runtime.

Java open source state machine frameworks are basically DSL-based implementations.

3. Open source state machine framework

We use three open source state machine frameworks to complete the process of SMS activity state flow.

3.1 Spring Statemachine

 enum ActivityState {
    NOT_START(0),
    DATA_PREPARING(1),
    DATA_PREPARED(2),
    DATA_PUSHING(3),
    FINISHED(4);
 
    private int state;
    private ActivityState(int state) {
        this.state = state;
    }
}
 
enum ActEvent {
    ACT_BEGIN, FINISH_DATA_CAL,FINISH_DATA_PREPARE,FINISH_DATA_PUSHING
}
 
@Configuration
@EnableStateMachine
public class StatemachineConfigurer extends EnumStateMachineConfigurerAdapter<ActivityState, ActEvent> {
    @Override
    public void configure(StateMachineStateConfigurer<ActivityState, ActEvent> states)
            throws Exception {
                states
                .withStates()
                .initial(ActivityState.NOT_START)
                .states(EnumSet.allOf(ActivityState.class));
    }
    @Override
    public void configure(StateMachineTransitionConfigurer<ActivityState, ActEvent> transitions)
            throws Exception {
                transitions
                .withExternal()
                .source(ActivityState.NOT_START).target(ActivityState.DATA_PREPARING)
                .event(ActEvent.ACT_BEGIN).action(notice())
                .and()
                .withExternal()
                .source(ActivityState.DATA_PREPARING).target(ActivityState.DATA_PREPARED)
                .event(ActEvent.FINISH_DATA_CAL).action(notice())
                .and()
                .withExternal()
                .source(ActivityState.DATA_PREPARED).target(ActivityState.DATA_PUSHING)
                .event(ActEvent.FINISH_DATA_PREPARE).action(notice())
                .and()
                .withExternal()
                .source(ActivityState.DATA_PUSHING).target(ActivityState.FINISHED)
                .event(ActEvent.FINISH_DATA_PUSHING).action(notice())
                .and() ;
    }
    @Override
    public void configure(StateMachineConfigurationConfigurer<ActivityState, ActEvent> config)
            throws Exception {
        config.withConfiguration()
                .machineId("ActivityStateMachine");
    }
    public Action<ActivityState, ActEvent> notice() {
        return context -> System.out.println("【变更前状态】:"+context.getSource().getId()+";【变更后状态】:"+context.getTarget().getId());
    }
 
   //测试类
   class DemoApplicationTests {
    @Autowired
    private StateMachine<ActivityState, ActEvent> stateMachine;
 
    @Test
    void contextLoads() {
        stateMachine.start();
        stateMachine.sendEvent(ActEvent.ACT_BEGIN);
        stateMachine.sendEvent(ActEvent.FINISH_DATA_CAL);
        stateMachine.sendEvent(ActEvent.FINISH_DATA_PREPARE);
        stateMachine.sendEvent(ActEvent.FINISH_DATA_PUSHING);
        stateMachine.stop();
    }
}

By overriding the three configure methods of the configuration template class, the state initialization, state transfer process and state machine declaration are completed in the form of streaming API, and the state machine of Java's internal DSL is realized. The external use of the state machine is triggered by the sendEvent event to promote the automatic flow of the state machine.

Advantage

• Spring Statemachine is an official product of Spring with a strong ecological community.
• The functions are very complete. In addition to supporting basic state machine configuration, it also has rich functional features such as nestable sub-state machines, zk-based distributed state machines, and external storage persistence.

defect

• Spring Statemachine saves the context-related properties of the current state machine within each statemachine instance, that is to say, it is stateful (this can be seen from triggering the state machine flow with only events as parameters), so the singleton pattern is used. State machine instances are not thread safe. To ensure thread safety, only a new state machine instance can be created each time through the factory mode. This method will affect the overall performance of the system in high concurrency scenarios.
• The code hierarchy is slightly complicated, and the cost of secondary development and transformation is high. In general scenarios, it is not necessary to use so many functions, and the look and feel is relatively heavy when used.

3.2 Squirrel Foundation

 public class SmsStatemachineSample {
    // 1. 状态定义
     enum ActivityState {
        NOT_START(0),
        DATA_PREPARING(1),
        DATA_PREPARED(2),
        DATA_PUSHING(3),
        FINISHED(4);
 
        private int state;
        private ActivityState(int state) {
            this.state = state;
        }
    }
 
    // 2. 事件定义
    enum ActEvent {
        ACT_BEGIN, FINISH_DATA_CAL,FINISH_DATA_PREPARE,FINISH_DATA_PUSHING
    }
 
    // 3. 状态机上下文
    class StatemachineContext {
    }
 
    @StateMachineParameters(stateType=ActivityState.class, eventType=ActEvent.class, contextType=StatemachineContext.class)
    static class SmsStatemachine extends AbstractUntypedStateMachine {
        protected void notice(ActivityState from, ActivityState to, ActEvent event, StatemachineContext context) {
            System.out.println("【变更前状态】:"+from+";【变更后状态】:"+to);
        }
    }
 
    public static void main(String[] args) {
        // 4. 构建状态转移
        UntypedStateMachineBuilder builder = StateMachineBuilderFactory.create(SmsStatemachine.class);
        builder.externalTransition().from(ActivityState.NOT_START).to(ActivityState.DATA_PREPARING).on(ActEvent.ACT_BEGIN).callMethod("notice");
        builder.externalTransition().from(ActivityState.DATA_PREPARING).to(ActivityState.DATA_PREPARED).on(ActEvent.FINISH_DATA_CAL).callMethod("notice");
        builder.externalTransition().from(ActivityState.DATA_PREPARED).to(ActivityState.DATA_PUSHING).on(ActEvent.FINISH_DATA_PREPARE).callMethod("notice");
        builder.externalTransition().from(ActivityState.DATA_PUSHING).to(ActivityState.FINISHED).on(ActEvent.FINISH_DATA_PUSHING).callMethod("notice");
 
        // 5. 触发状态机流转
        UntypedStateMachine fsm = builder.newStateMachine(ActivityState.NOT_START);
        fsm.fire(ActEvent.ACT_BEGIN,  null);
        fsm.fire(ActEvent.FINISH_DATA_CAL, null);
        fsm.fire(ActEvent.FINISH_DATA_PREPARE, null);
        fsm.fire(ActEvent.FINISH_DATA_PUSHING, null);
     }
}

squirrel-foundation is a lightweight state machine library designed to provide a lightweight, highly flexible, extensible, easy-to-use, type-safe, and programmable state machine implementation for enterprise use.

Advantage

• Consistent with the target concept, compared with Spring Statemachine, it does not depend on the spring framework, and the design and implementation are more lightweight. Although it is also a stateful design, the cost of creating a state machine instance is less, and the function is more concise. It is relatively suitable for two secondary development.
• Corresponding documents and test cases are also abundant, making it easy for developers to get started.

defect

• Too much emphasis is placed on the concept of "convention over configuration", and many default processes, such as actions after state transitions are invoked through method names, which are not conducive to operation management.
• Community activity is not high.

3.3 Cola State machine

 /**
 * 状态机工厂类
 */
public class StatusMachineEngine {
    private StatusMachineEngine() {
    }
    private static final Map<OrderTypeEnum, String> STATUS_MACHINE_MAP = new HashMap();
 
    static {
        //短信推送状态
        STATUS_MACHINE_MAP.put(ChannelTypeEnum.SMS, "smsStateMachine");
        //PUSH推送状态
        STATUS_MACHINE_MAP.put(ChannelTypeEnum.PUSH, "pushStateMachine");
        //......
    }
 
    public static String getMachineEngine(ChannelTypeEnum channelTypeEnum) {
        return STATUS_MACHINE_MAP.get(channelTypeEnum);
    }
 
   /**
     * 触发状态转移
     * @param channelTypeEnum
     * @param status 当前状态
     * @param eventType 触发事件
     * @param context 上下文参数
     */
    public static void fire(ChannelTypeEnum channelTypeEnum, String status, EventType eventType, Context context) {
        StateMachine orderStateMachine = StateMachineFactory.get(STATUS_MACHINE_MAP.get(channelTypeEnum));
        //推动状态机进行流转，具体介绍本期先省略
        orderStateMachine.fireEvent(status, eventType, context);
    }
 
/**
 * 短信推送活动状态机初始化
 */
@Component
public class SmsStateMachine implements ApplicationListener<ContextRefreshedEvent> {
 
    @Autowired
    private  StatusAction smsStatusAction;
    @Autowired
    private  StatusCondition smsStatusCondition;
 
    //基于DSL构建状态配置，触发事件转移和后续的动作
    @Override
    public void onApplicationEvent(ContextRefreshedEvent contextRefreshedEvent) {
        StateMachineBuilder<String, EventType, Context> builder = StateMachineBuilderFactory.create();
        builder.externalTransition()
                .from(INIT)
                .to(NOT_START)
                .on(EventType.TIME_BEGIN)
                .when(smsStatusAction.checkNotifyCondition())
                .perform(smsStatusAction.doNotifyAction());
        builder.externalTransition()
                .from(NOT_START)
                .to(DATA_PREPARING)
                .on(EventType.CAL_DATA)
                .when(smsStatusCondition.doNotifyAction())
                .perform(smsStatusAction.doNotifyAction());
        builder.externalTransition()
                .from(DATA_PREPARING)
                .to(DATA_PREPARED)
                .on(EventType.PREPARED_DATA)
                .when(smsStatusCondition.doNotifyAction())
                .perform(smsStatusAction.doNotifyAction());
        ...(省略其他状态)
        builder.build(StatusMachineEngine.getMachineEngine(ChannelTypeEnum.SMS));
    }
 
   //调用端
   public class Client {
     public static void main(String[] args){
          //构建活动上下文
          Context context = new Context(...);
         // 触发状态流转
          StatusMachineEngine.fire(ChannelTypeEnum.SMS, INIT, EventType.SUBMIT, context);
      }
   }
}

Cola Statemachine is a state machine framework in Alibaba's COLA open source framework. The biggest difference from the previous two is: stateless design - when triggering the state machine flow, the current state needs to be used as an input parameter, and there is no need to keep it in the state machine instance. The current state context message has only one state machine instance, which directly guarantees thread safety and high performance.

Advantage

• Lightweight stateless, secure, high performance.
• The design is simple and easy to expand.
• Community activity is high.

defect

• Advanced functions such as nesting and parallelism are not supported.

3.4 Summary

The three open source state machine frameworks are compared as follows:

Systems that wish to directly utilize the capabilities of open source state machines can select appropriate models according to their own business needs and process complexity.

4. Marketing Automation Business Case Practice

4.1 Design selection

The business features of vivo marketing automation are:

• There are many types of operational activities, large business traffic, relatively simple processes, and high performance requirements.
• Processes change frequently, and business statuses need to be added frequently, and rapid configuration and changes need to be supported.
• After the state is triggered, there will be a variety of different business operations, such as message reminders after the state changes, business processing after the state ends, etc., which need to support asynchronous operations and facilitate expansion.

In view of the above business characteristics, in the actual project development, we are based on the implementation plan of the open source state - development based on the internal DSL. At the same time, it draws on the characteristics of the above open source frameworks, and chooses a lightweight design that is stateless, high-performance, simple in function, and supports asynchronous execution of actions.

• Stateless high performance : To ensure high performance, using a stateless state machine design, only one state machine instance is needed to operate.
• Simple function : the principle of minimal design, only the core design is retained, such as event triggering, basic state flow, subsequent operations and context parameter processing.
• Asynchronous execution of actions : For asynchronous business processes, use thread pools or message queues for asynchronous decoupling.

4.2 Core Process

• Following the design of the internal DSL streaming interface of the open source state machine, the state machine definition is scanned when the application starts;
• Create an asynchronous processing thread pool to support the post-action of the business;
• Parse the DSL configuration of the state machine and initialize the state machine instance;
• Build an execution context to store instances of each state machine and other execution process information;
• When the state machine is triggered, it automatically matches the transition process according to the trigger condition and the current state, and promotes the flow of the state machine;
• Perform post-sync/async processing operations.

(Figure 4-1: Core Process Design)

4.3 Practical thinking

1) State machine configuration visualization, combined with external DSL (such as JSON, stored in the database) to support faster configuration.

2) Currently, only the simple flow of status is supported, and the flow interface extension point is added in the flow process to deal with complex scenarios that may appear in the future.

V. Summary

A state machine is composed of events, states, and actions. The relationship between the three is: an event triggers the transition of the state, and the transition of the state triggers the execution of subsequent actions. Using a state machine for system state management can improve business scalability and cohesion. State machines can be implemented using conditional branching, state patterns, and DSLs, where more expressive DSLs are also the way many open source state machines are implemented. You can make appropriate selections based on the characteristics of the open source state machine and your own project requirements, or you can customize the state machine components based on the previous solutions.

This article is the third in a series of special articles on "Demystifying Marketing Automation Technology". The series of articles reviews:

"Deciphering Marketing Automation Technology | Beginning"

"How Design Patterns Improve the Business Expansion of vivo Marketing Automation | Engine 01"

Later, we will continue to bring other content of the series of special articles. Each article will provide a detailed analysis of the technical practice in it, so stay tuned.

Author: vivo Internet Server Team - Chen Wangrong