Vivo marketing automation technology decryption｜Opening

1. Overview of marketing automation

1.1. What is marketing automation

Marketing automation refers specifically to the marketing department or organizational design of software platforms and technologies can be more effective online multi-channel marketing and automate repetitive tasks . Marketing and sales staff, then explain, stored and executed by the IT system by operating the standard-setting tasks and processes to increase efficiency and reduce human error .

The purpose of the marketing automation platform is to simplify sales and marketing organizations by replacing high-touch, repetitive manual processes with automated solutions.

The marketing automation platform is a dashboard used by marketers to plan, coordinate, manage and measure all online and offline marketing activities. It is usually used in conjunction with user lifecycle marketing strategies to closely manage and nurture the generated potential customers, aiming to convert potential customers into customers.

Marketing automation is a subset of customer relationship management (CRM) that focuses on the definition, segmentation, scheduling, and tracking of marketing activities. The use of marketing automation makes processes that originally required manual execution more efficient and makes new processes possible.

Marketing automation can be defined as a process in which technology is used to automate multiple repetitive tasks that are regularly performed in marketing activities. Tools that allow individuals to design, execute, and automate time-limited marketing workflows can be referred to as marketing automation platforms.

The marketing automation platform allows marketers to manage complex omni-channel marketing strategies through a single tool to automate and simplify customer communication. Marketing automation provides great help in the areas of potential customer generation, segmentation, potential customer training and potential customer scoring, relationship marketing, cross-selling and up-selling, retention, and marketing return on investment measurement. Effective marketing automation tools use data from separate or integrated CRMs to understand customer influence and preferences.

1.2. Why develop a marketing automation platform

1.2.1 Business problems and pain points

• It is difficult to grasp the pain of marketing timing, it is impossible to effectively identify the active and automatic operation timing, it is difficult to achieve timely contact, and communication opportunities are lost.
• For different delivery channels and users, if all users see exactly the same content, the operation effect will definitely be poor, which will seriously affect the user’s reputation.
• The pain of being unable to operate manually, many operating scenarios need to be put into execution in real time, and manuals cannot monitor and operate at all times.
• There is a single user access channel, which cannot effectively integrate all access channels, resulting in a waste of access.
• It is difficult to measure the effect of an activity, and it is impossible to effectively attribute the conversion effect. It is not known whether the activity is good or bad, and ROI cannot be measured.

1.2.2 Advantages of Marketing Automation Platform

• Improve management efficiency, standardize the marketing process, automatically supervise and evaluate the marketing process, to solidify marketing SOPs, ensure marketing quality, improve service quality, replace a large number of repetitive tasks, and save labor costs.
• Improve marketing conversion efficiency, improve marketing effects in the short-term, and increase user LTV in the long-term.
• To improve user experience satisfaction, through professional creative solutions, with reasonable rights and interests resources, carry out marketing activities that users like, avoid marketing activities to disturb users, and improve NPS.

1.3. What are the challenges of marketing automation platform construction

1) Multi-department collaboration to break data islands.

Build a massive big data marketing data warehouse and provide powerful big data OLAP technology to meet the needs of large-scale data processing.

2) Provide a rich library of marketing strategy plans to support changeable marketing scenarios, increase reach conversion rate, and achieve the ultimate goal of refined operation and smart marketing.

It is necessary to develop a flexible rule engine to realize intelligent grouping, intelligent scheme design, intelligent matching and automatic delivery capabilities.

3) High availability requirements of 99.99% or more.

As a connecting middle station for data and business, the technical architecture must ensure the high availability of the system to support the goal.

The following is an overall introduction to the overall business and technical architecture of the vivo marketing automation platform.

2. Product architecture of vivo marketing automation platform

The vivo marketing automation platform is mainly divided into four major business modules:

2.1. Crowd management module

The main function of the crowd management module is to provide a target audience for the marketing automation system and complete the refined grouping. There are three grouping methods: manual grouping , regular grouping and intelligent grouping .

For the B-end marketing automation system, the crowd management module also includes lead management functions. The clue here is potential users. The lead management function also provides lead scoring grading to more effectively complete marketing conversion.

The technical challenges of the crowd management module are:

• Provide massive big data OLAP technology for real-time crowd circle selection;
• Provide ultra-large-scale data processing technology.

The main functions of the crowd management module are: clue management, identity management, potential customer management, crowd management, crowd circle selection, crowd sampling, crowd expansion, crowd orientation, and passenger flow insight.

2.2. Marketing Strategy Module

The main function of the marketing strategy module is to provide a strategy plan library for the marketing automation system. This strategy library refers to the marketing strategy and delivery process established for users. For example, for one week of purchase, it is a push game technique for new purchasers; for users entering the order page, one day's unpaid settlement will push the user's coupon information to help the final conversion; consumption 3 times within seven days, a 50% discount coupon will be given.

The marketing plan is the accumulation of experience in the marketing automation system and is one of the core assets. It can standardize and guide marketing behavior and establish a marketing SOP. The core function of the marketing strategy module is program library management and program matching. The scheme is mainly divided into static scheme and dynamic scheme. In the initial stage of the marketing automation system, in order to verify the product form and value, static solutions will be adopted. When the system is gradually improved, it will gradually provide dynamic solution capabilities. Eventually, static solutions and dynamic solutions will coexist in the system and are compatible with each other. Matching plan refers to matching the best marketing strategy for users. When users can match multiple solutions, the system needs to filter out the best solution based on user experience, cost and other factors.

The marketing strategy module also includes the material management function, which manages the text, pictures, videos, applets, h5 pages and app content materials in a unified manner.

The technical challenges of the marketing strategy module include: a powerful rule engine, a flexible workflow engine and system modeling.

The main functions of the marketing strategy module are: program management, program design, event management, content management and user journey design functions.

2.3. Reach the release module

The main function of the touch release module is to integrate the touch channels of vivo, open up the full link of touch, provide a one-stop touch platform, and provide the effect data recovery function.

The channel management function mainly provides various contact access configuration and content template configuration. Through the channel integration function, to reach the optimal mechanism in the whole, choose the appropriate channel and replacement strategy. The marketing automation system also needs to consider the interruption of marketing activities to users and avoid negative word-of-mouth, so it needs to provide anti-harassment functions, mainly through frequency control settings and black and white orders.

reaches the technical challenges of the delivery module: real-time delivery execution engine, high-performance and high-availability delivery requirements.

The main functions of the contact delivery module include: channel management, channel integration, performance data evaluation and user anti-harassment functions.

2.4. Measurement Center Module

The main function of the measurement center module is to provide operational index reports for the effects of marketing activities and evaluate operational effects. Record the delivery process data and conversion result data, analyze and count scientifically set measurement indicators, evaluate whether the input-output ratio of marketing activities is expected, determine whether the core indicators of operational effects have reached the target, help users understand the target achievement status, and discover marketing There are problems in the entire link of the event, optimize the next marketing plan. The core indicators of the measurement center module are user activation, retention, churn, reach, store arrival, and repurchase.

Three, vivo marketing automation platform system architecture

3.1 System architecture

[Gateway layer] Nginx load balancing;

[Service layer] The are crowd circle selection services, activity configuration services, program execution services, delivery services, and measurement services.

• Crowd circle selection: provide functions of crowd creation, crowd management, crowd calculation, and crowd orientation;
• Activity configuration: Responsible for the management and configuration business of the docking operation background, and provide basic configuration capabilities and information interfaces;
• Program execution: Responsible for the actual execution process of the activity, including activity triggering and data processing;
• Delivery service: abstract reach capabilities, and provide services with functions to reach external channels in a unified manner;
• Measurement services: Provide visualization capabilities of operational measurement index reports;

These five major services can be split independently. Considering the complexity of the system, they can be defined in a modular manner in the early stage.

[computing layer] mainly includes basic data services, automation program execution and channel access services

• Data service: Provide data insight, crowd calculation, intelligent grouping and crowd orientation functions.
• Marketing automation engine: Obtain activity configuration information, perform index calculation, crowd matching and rule matching according to the marketing plan, and finally output the results for distribution.
• Channel reach service: the aggregation service layer of omni-channel contact points, docking with the sending and data recovery interfaces of external contact services including SMS, push, WeChat, etc. Processing process: prepare to put materials, address users, conduct anti-harassment frequency control, and finally reach users.

[Storage layer]

• It mainly uses MySQL to store business data, while using Redis as a cache service, and ES as a tool for data query.
• Another key big data application is the Presto query engine, as the engine for massive data analysis and query (due to space reasons, the scenario application practice of the Presto engine will be analyzed in depth in subsequent articles).

[Public component]

• Monitoring system: system-level monitoring and business index monitoring.
• Rule engine: The rule engine is a key component for processing complex event rules. It uses its powerful configuration and expansion capabilities to support a variety of activity rules (the application practice of the rule engine will also be analyzed in depth in subsequent articles).

3.2. System Design

3.2.1 Marketing strategy engine design

Customize each marketing business process node through workflow engine components to support highly scalable activity business. The core process includes: creation of operational activities (configuration trigger timing, activity users, content configuration), approval of operational activities, and review of operational activities implement.

In addition, for the specific activity configuration, activity approval and activity execution node logic implementation within the business process, design patterns and design principles are used for componentization and abstract processing to achieve the goal of higher scalability.

3.2.2 Integrated management of marketing omni-channel touchpoints

By configuring different push strategies for multi-dimensional combined delivery, complementary channels can be configured in the same event to increase the coverage of reaching users. For example, for a certain group of active users, first use more in-depth business channels (for example: WeChat channels) for event delivery. Users who have not linked WeChat accounts or who fail to reach WeChat due to other reasons should use channels with higher coverage (such as: SMS, each user uses the mobile phone number as the account number) to supplement the reach and increase the overall reach rate.

Multi-channel data needs to be unified to analyze the overall effect of the activity, and to control the frequency of contact to prevent harassment. Another key point is the need to improve the reach performance of a single channel (the current omni-channel daily release of users can reach tens of millions), and at the same time do a good job of resource control and isolation between different channels.

3.2.3 Reaching of real-time marketing scenarios

Through business real-time event stream processing, the rules of different activities are matched to achieve real-time reach effects.

The access integration of business data and log data is used as the basis, the trigger data -> data index calculation -> rule matching → result output.

Take [the user sends a text message reminder within 10 minutes of placing an order] for example:

1. The business system sends the order data to the marketing automation platform through RabbitMQ.

2. The marketing automation platform uses stored user information and order event configuration to complete user data (surname, mobile phone number).

3. Use the rule engine to match the event of [Order 10 minutes].

4. After the matching is passed, the result data will be forwarded to the contact system in a unified format for SMS transmission.

3.3. High-availability construction

1) Monitoring & Alarm System

Including system-level monitoring and business indicator monitoring. With the help of infrastructure capabilities, system-level monitoring and alarming of hosts, DBs, containers, and domain names are carried out; at the same time, the operating status and key indicators of the business are set up to monitor in real time.

2) Smooth release

Ensure smooth upgrade and release of applications without affecting the integrity of tasks and data reached. The rise of microservices, DevOps, and cloud native emphasizes that software can be delivered anytime, anywhere. In the publishing process, it is necessary to pay attention to the impact on the system, and publish as non-destructively as possible. For example, when it is being published, the SMS is being delivered at the same time. At this time, it must be ensured that the publishing process does not affect the data integrity of the SMS delivery. The restart hook function performs the business processing of the port restart, and it can also be retried by saving the business data state.

3) Current limit

Limit the overloaded traffic to prevent normal business from being affected. During the operation of the marketing system, marketing hotspots will inevitably be encountered, but when large-traffic activities are passively triggered or actively launched activities to large-cap users, it is necessary to evaluate the system's affordability, and set adjustable thresholds under the affordability for current limiting control. .

4) Compensation for retry and data idempotence

As a marketing platform, there are many system businesses involved, and the core data delivery interface has a failure retry mechanism. Correspondingly, data idempotence must be guaranteed. For example, a certain activity is only allowed to push tasks to the same user once. In this case, the idempotent anti-re-verification can be performed based on the activity id + user id as the unique primary key of the business.

Four, summary

The marketing automation system has been widely used in the vivo marketing field and nationwide agency companies. It is mainly used in user life cycle operations, new product launch marketing, major nodes, membership days and other refined marketing activities. Business parties generally give feedback on their marketing efforts. Large, save a lot of labor costs, improve marketing efficiency, and provide obvious value.

This article is the beginning of a series of special articles on "Marketing Automation Technology Decryption", which gives an overall introduction to marketing automation business and technology. Of course, this is just the beginning. There are still many practical experience summaries in the use and scene implementation of marketing automation technology to share with you. In the follow-up, we will continue to bring the following "Marketing Automation Technology Decryption" series of feature articles:

1. "Decryption of Marketing Automation Technology | Engine Chapter-Design Patterns and Application Practice"

2. "Marketing Automation Technology Decryption｜Engine Chapter-State Machine Application Practice"

3. "Marketing Automation Technology Decryption｜Engine Chapter-Workflow Engine Application Practice"

4. "Marketing Automation Technology Decryption｜Engine Chapter-Real-time Event Engine Application Practice"

5. "Marketing Automation Technology Decryption｜Engine Chapter-Rule Engine Application Practice"

6. "Decryption of Marketing Automation Technology | Channel Chapter"

7. "Marketing Automation Technology Decryption｜Data Article"

......

Each article will give a detailed analysis of the technical practices inside, so stay tuned.

Author: vivo internet server team-Liao Yi