Cutting-edge Sharing｜Alibaba Cloud Database Solution Architect Wang Hongyu: In-depth application and business value of cloud-native data warehouse AnalyticDB in the retail industry

Introduction to This content is for the 2021 Cloud Home Conference-Cloud Native Data Warehouse AnalyticDB Technology and Practice Summit Sub-forum, Alibaba Cloud Database Solution Architect Wang Hongyu on "The deep application and business value of cloud native data warehouse AnalyticDB in the retail industry "To share.

more cutting-edge sharing, click Conference video playback link to get it.

This article will introduce the best practices based on the cloud-native data warehouse AnalyticDB MySQL through three parts.

1. The development trend of the retail industry

2. The core capabilities of AnalyticDB

Third, the application of AnalyticDB in retail

1. The development trend of the retail industry

From the earliest supermarkets and department stores to the current e-commerce and new retail, they have been developed around three core elements: people, goods, and markets. Traditional retail mostly starts from the field: first, retail places are created, and then users come to consume; consumer behavior starts at the retail place and ends at the retail place.

But with the application of digitization and informatization, the relationship between people, goods, and markets is being reconstructed by data and profound changes have taken place: retail has returned to the "people-oriented" concept-where is the user's needs, retail Wherever it happens, such as unmanned containers in the office, shared charging treasures, etc.; at the same time, a three-dimensional network with "people as the core" has gradually formed-transaction behavior has broken through the limitations of time and space, and can occur anytime, anywhere, and consumption The life cycle of the behavior will also be longer.

The digitalization of the retail industry is embodied in three places:

First, the digitization of "people". Whether it’s an offline Wanda store or an online Taobao mall, its essence lies in attracting user traffic—attracting users into the store, then analyzing user traffic, and finally consuming user traffic, so the digitization of people actually lies in the analysis and consumption of user traffic. .

Merchants can obtain user data through different channels, such as data from their own e-commerce platform, Weibo fan data, or WeChat public account Moments, etc. After data collection is completed, businesses will use different data mining algorithms to analyze user portraits from various dimensions, extract user behavior tags for classification, and finally formulate different marketing plans for different customer groups. How to achieve accurate analysis of the crowd will have a very important impact on retail, such as passenger flow will determine the location of the store.

Second, the digitization of "goods". The digitization of goods is mainly carried out around the optimization of the entire supply chain, including the digitization of various links such as multi-channel distribution/ordering, order management, and fulfillment delivery. Global integration and management will bring some challenges to the retail industry: including what distribution strategy/sales strategy is adopted between online/offline multiple channels, how to manage inventory uniformly, how to achieve rapid delivery, how to improve repurchase, etc. Wait.

For example, the speed from delivery to delivery to consumers on Double 11 each year is getting faster and faster. Behind this is the magical power of the digitalization of goods: before Double 11, Taobao/Tmall/JD.com and other e-commerce platforms, Will analyze the consumer behavior of users in the recent period, and make advance judgments-analyze which products have a high repurchase rate, which products have geographic attributes, etc., and then place these products closer to consumers in advance The front warehouse; after the consumer places an order, the goods will be shipped directly from the front warehouse. At the same time, in the logistics industry, the general electronic face order system also digitizes all aspects of logistics, which also greatly improves the speed of goods circulation.

Third, the digitization of the "field". Mainly compare the respective advantages/disadvantages between online and offline channels, and then use each other's advantages to complete the reconstruction of information flow and capital flow. Offline stores can fully experience the product, but the overall cost is much higher than online stores.

So many companies combine offline stores with online e-commerce stores, such as Xiaomi Home & Xiaomi Mall, TATA Mumen's offline experience store & Tmall flagship store, etc., which have greatly improved the efficiency.

The digitalization of the retail industry has realized the unified management of omni-channel merchandise/orders, and has accumulated a large amount of user data, making the marketing effect more intuitive, but it has also led to a rapid increase in the amount of data. How to achieve real-time/accurate analysis of user data, product reports, and timely and rapid display of marketing effects in massive data are also problems and challenges faced by retailers.

Second, the core capabilities of AnalyticDB

The above is the architecture diagram of the native data warehouse AnalyticDB in the entire data link.

Bottom-up, data such as structured/unstructured data, log data, and file data on object storage can be aggregated into AnalyticDB in real time or offline through different tools; then use the performance advantages of AnalyticDB's complex queries to complete the data Statistical analysis; finally with the help of open source or commercial BI display tools, or business programs, graphical or interactive display. Of course, you can also use data development/scheduling tools, such as DMS, DataWorks, to realize data ETL batch processing, and realize integrated data warehouse on/offline.

The core capabilities of AnalyticDB are mainly embodied in three areas: fast query performance, real-time analysis and easy-to-use.

• AnalyticDB uses a new generation of ultra-large-scale MPP+DAG fusion engine, and uses row-column mixed storage, intelligent indexing and other technologies to greatly improve query performance. Complicated SQL query speed is more than 10 times faster than traditional relational databases, and it is also several times faster than traditional data warehouse products.
• With the help of the DTS real-time synchronization tool, changes to the business database can be transmitted to ADB in time. From data change to analysis to display, the entire link delay is in the second level.
• It is highly compatible with MySQL and PG protocols, and can be easily used through standard SQL, common BI tools, and ETL tool platforms, which greatly reduces the construction and maintenance costs of data warehouses.

As a new OLAP product, AnalyticDB usually has two common application scenarios:

• Interactive BI analysis. For example, the Tmall Double 11 large screen covers statistics related to total transaction volume, category TOP, and region. Advantages: high query performance, can reach trillion-level data analysis sub-second response, query speed is about 100 times that of MySQL.
• l Log analysis. Such as game operation analysis and IT operation and maintenance log analysis. Advantages: Achieve the fusion analysis of structured and unstructured data, and the separation of cold and hot at the same time greatly reduces storage costs.

Third, the application of AnalyticDB in retail

How does AnalyticDB help customers in the retail industry increase their business value? Let's look at a few customer cases.

The first case comes from the customer cloud. Keruyun is a service provider that provides SAAS solutions to local life service businesses such as catering, retail, and beauty.

The client has three main demands:

• The report is displayed in real time. Traditional data warehouses can only display T+1, which may result in merchants being able to check the operation status the next day, which in turn causes delays in replenishment and resource allocation and affects normal sales.
• Profile analysis value-added services. Keruyun’s merchants hope that they can provide more accurate profile analysis services, so that they can provide more intimate catering services for different target groups, such as couple packages, economic packages, discount coupons, etc.
• Stability and scalability. Such as Valentine's Day, Chinese Valentine's Day, Christmas and other holiday dining peaks, we need to ensure the smoothness of the system.

This is the architecture diagram after the architecture upgrade.

PolarDB MySQL replaces traditional MySQL, bears business traffic, and has extreme flexibility.

DTS synchronizes the data changes in the business database to AnalyticDB in real time to realize the decoupling and real-time synchronization of the business database and the analysis database.

AnalyticDB helps customers realize real-time report analysis, complex interactive query and user profile analysis and other functions.

Through this architecture upgrade, AnalyticDB has helped Keruyun expand its business boundaries and found new revenue growth points: the launch of the merchant report VIP package, the report update is reduced from the day to the hour level; at the same time, the user portrait precision marketing service has been developed. This new feature adds hundreds of millions of revenue to Keruyun every year. At the same time, during the peak dining seasons of Tanabata, National Day, Christmas and other holidays, the system runs very smoothly without any lag.

The second case comes from Beijing Fengchuang Technology. Beijing Fengchuang Technology China's enterprise-level marketing integrated management SaaS platform. It owns multiple product platforms such as marketing activity management platform, CRM user relationship management platform, community operation system, and precision marketing delivery platform.

Mainly face several problems:

• Poor query performance. The amount of table data is large, the amount of data in a single table is over 100 million or even billions, and there are many scenarios for multi-table association/multi-dimensional interactive query. Moreover, advertisers have very high requirements for the timeliness of marketing presentation.
• The traditional data warehouse structure is complicated. There are many components involved, long data links, and high cost of personnel learning and operation and maintenance.
• Extensibility. It can carry the growth of data in the next 3-5 years, without the need to upgrade the architecture again.

Combining business scenarios, the PolarDB-X+DTS+AnalyticDB solution is adopted: distributed PolarDB-X is used as a sub-database and sub-table to undertake high business concurrency; data is transmitted to AnalyticDB in real time through DTS; at the same time, AnalyticDB can also directly read data on OSS Perform a joint query. In this way, a data analysis platform for data aggregation, data cleaning, ETL calculation and real-time query services is constructed.

After the architecture is completed, the introduction of AnalyticDB enables the performance of multi-dimensional analysis and query to be returned in seconds, and the display of marketing effects is more timely. At the same time, the rapid flexibility of AnalyticDB and the separation of hot and cold data make the overall cost more controllable.

The third case comes from Shanghai FenShang Network, a leading brand of domestic flower e-commerce, which has created a daily flower subscription model of "online subscription + direct delivery from the origin + value-added service".

The main problem is: business database and analysis database use traditional MySQL, analysis scenarios such as order, product flow, purchase, business conversion rate, product sold out alarm, etc., the query speed is slow or even can not be queried; business development is very fast , The amount of data has grown rapidly; the technical team is familiar with the MySQL ecology, and the cost of learning more traditional data warehouse components is high; in addition, it is to consider the scalability of the system in the case of further data growth in the future.

Later, OLAP analysis was placed on AnalyticDB: using the excellent query performance of AnalyticDB, the report and BI analysis speed was increased by 2-10 times, and the overall business responsiveness and customer service experience were also greatly improved. At the same time, using ADB's data cold and hot separation and resource group flexibility functions, higher scalability and flexibility, IT expenditure costs are reduced by more than 30%.

There are more customers in the retail industry, such as Feihe, Juranzhijia, Business Staff, etc., are also using AnalyticDB to carry complex report statistics and interactive analysis scenarios, and tap more business value through digital transformation.

Copyright Notice: content of this article is contributed spontaneously by Alibaba Cloud real-name registered users, and the copyright belongs to the original author. The Alibaba Cloud Developer Community does not own its copyright and does not assume corresponding legal responsibilities. For specific rules, please refer to the "Alibaba Cloud Developer Community User Service Agreement" and the "Alibaba Cloud Developer Community Intellectual Property Protection Guidelines". If you find suspected plagiarism in this community, fill in the infringement complaint form to report it. Once verified, the community will immediately delete the suspected infringing content.