Heavyweight｜Alibaba Cloud releases "one-stop agile data warehouse solution" to achieve integrated data analysis capabilities for warehouses (including dry goods PPT download)

Introduction to Cloud released a one-stop agile data warehouse solution. The solution combines the one-stop data management platform DMS and the cloud native data warehouse AnalyticDB (hereinafter referred to as ADB) to truly realize the integrated technical architecture of the warehouse, providing online data real-time warehousing, T+1 periodic snapshots, on-demand warehouse building, etc. Ability, data latency is as low as seconds, and continues to empower business online, allowing enterprises to release the maximum value of online data. Through low-code operations, Alibaba Cloud's one-stop agile data warehouse solution greatly reduces the difficulty of building real-time data warehouses and the threshold of data processing. At the same time, it can support various high-frequency and dynamic real-time analysis scenarios and needs of enterprises, and help users crack Real-time data warehouse construction problems accelerate the digital transformation of enterprises.

Cloud, Smart Future" 1615d73ab4116c —— Cloud Database Innovation and the 3rd Database Performance Challenge Final Award Ceremony has been successfully concluded. More dry content is welcome to watch the live replay of the summit.

Summit live playback 📎 https://developer.aliyun.com/live/247301

Dry goods PPT download📎 https://developer.aliyun.com/topic/download?id=7986

On September 26, the "Data Convergence Cloud · Smart Control of the Future"-Alibaba Cloud Database Innovation Summit, co-sponsored by Alibaba Cloud and Intel, was held in Beijing. At the summit, Alibaba Cloud released a one-stop agile data warehouse solution. The solution combines the one-stop data management platform DMS and the cloud native data warehouse AnalyticDB (hereinafter referred to as ADB) to truly realize the integrated technical architecture of the warehouse, providing online data real-time warehousing, T+1 periodic snapshots, on-demand warehouse building, etc. Ability, data latency is as low as seconds, continuously empowering business online, enabling enterprises to release maximum value of online data.

Through low-code operations, Alibaba Cloud's one-stop agile data warehouse solution greatly reduces the difficulty of building real-time data warehouses and the threshold of data processing. At the same time, it can support various high-frequency and dynamic real-time analysis scenarios and needs of enterprises, and help users crack Real-time data warehouse construction problems accelerate the digital transformation of enterprises.

Real-time data storage, second-level analysis of business data

Nowadays, data has become one of the indispensable core means of production in the process of enterprise development. With the deepening of digitization of enterprises, how to improve productivity through data is the direction of enterprises' continuous exploration. As the core carrier of data integration, development and service, the data warehouse plays a key role in it.

Traditional data warehouses generally build offline data warehouses based on T+1 data integration to support various analysis and services of the enterprise. Traditional solutions will not only affect the stability of online business, but also difficult to support the real-time needs of enterprises. As enterprises have higher and higher requirements for the timeliness of data value realization and applications, real-time data warehouses have emerged.

Alibaba Cloud's one-stop agile data warehouse solution is based on many years of real-time data warehouse construction experience of Alibaba Group, combined with the one-stop data management platform DMS and cloud-native data warehouse ADB, and supports real-time storage of business data + addition, deletion, modification, and query based on zipper tables. T+1 periodic snapshots and other functions fully meet the needs of enterprise application scenarios for real-time analysis of production data, such as large promotional screens, monitoring reports, precision marketing, interactive operation analysis, etc.

Traditional real-time data warehouse construction solutions are usually difficult to meet the demand for real-time warehousing of multi-source heterogeneous data. At the same time, data processing links are long and efficiency cannot be guaranteed. Compared with traditional solutions, Alibaba Cloud's one-stop agile data warehouse solution has the following four core advantages:

1. The impact on the business side is small, and the normal operation of the business side will not be affected by data aggregation and real-time processing, and the CPU and memory usage are less than 5%;

2. The transaction sequence and data accuracy are guaranteed, and the processing link is short. It supports online data real-time processing and settlement, which is more efficient. The data transmission efficiency is 100m/s, and the data delay is within 10 seconds;

3. Support complex real-time data processing and calculation logic;

4. Low-code operation can greatly reduce the difficulty of real-time data warehouse construction, improve construction efficiency, and support various real-time scenarios in the process of enterprise digital transformation.

DMS integrated data transmission service DTS, can support dozens of data sources of stock data + incremental update data, real-time synchronization to ADB, to ensure that the real-time data warehouse side data is consistent with the production side. In addition, DTS supports real-time data processing and processing during real-time data storage and data integration, helping companies reduce costs and increase efficiency.

Periodic snapshots, zero interruption in business

In addition to real-time statistical analysis scenarios, enterprises need to build periodic full snapshots to meet the needs of periodic data analysis. The periodic full integration solution of the traditional data warehouse will have a stable impact on the production business, the full integration has poor timeliness, and cannot meet the business demands of customers for data backtracking at any point in time.

For T+1 periodic integration scenarios, the one-stop agile data warehouse solution supports T+1 full data snapshots based on zipper tables. Users can generate full or incremental snapshots of various cycles on demand through a few simple steps. . In addition, the business can also perform data backtracking at any point in time as needed to quickly resolve data abnormalities.

A real-time data warehouse construction case of a bank

Take a bank as an example. The bank adopts a one-stop agile data warehouse solution to build a real-time data warehouse to support efficient and customized analysis and decision-making needs of various business lines. DMS integrates DTS non-intrusive real-time log analysis capabilities and real-time data storage, which greatly reduces the resource consumption of business systems during data synchronization, zero interference with business operations, and greatly reduces the difficulty of real-time data warehouse construction. In addition, the solution helped the bank quickly generate a snapshot table at a T+1 (hour) frequency. It only takes 3 minutes to take a snapshot of tens of millions of data. Combined with ADB's millisecond-level interactive analysis capabilities for massive data, it further improves real-time data warehouse analysis. Support efficiency of decision-making.

Chen Changcheng, head of the ecological tools department of Alibaba Cloud Database Product Division

" Cloud senior technical expert and head of the ecological tools department of the database product division Chen Changcheng said: "The goal of one-stop data management DMS is to help enterprises maximize the value of online data. Enterprises use one-stop data management DMS+ADB It can quickly build a real-time data warehouse and realize the integrated technical architecture of the warehouse. Compared with traditional solutions, it can effectively solve a variety of online data assetization problems, and realize on-demand warehouse building, data concentration, and agile development.

With the integrated DMS data transmission and processing link of the warehouse, the data stream ETL link is greatly shortened, maintenance costs are reduced, and the flexible task scheduling and productization support T+1 scenarios through data development can reduce the data processing threshold and quickly play The value of enterprise online data. "

Zhan Chaoqun, Head of OLAP Product Department of Alibaba Cloud Database Product Division

" Cloud researcher and head of the OLAP product department of the database product division Zhan Chaoqun said: "The cloud-native data warehouse AnalyticDB takes cloud native, database and big data integration as the core, and supports a data multi-computing scenario, full link Features such as real-time data, support structured and unstructured data fusion analysis, can realize ultra-large-scale real-time addition, deletion, modification, and on-demand complex analysis with high concurrency and low latency, helping businesses realize agile, real-time, and intelligent digital innovation .

Alibaba Cloud's one-stop agile data warehouse solution, combined with one-stop data management DMS+ADB, provides enterprises with agile data warehousing, ETL management, task scheduling, asset management and other capabilities to maximize the value of enterprise data assets in the future. We will also introduce more enterprise-level capabilities, welcome everyone to pay attention. "

At present, Alibaba Cloud's one-stop agile data warehouse solution has been widely used in finance, digital government, retail, pan-Internet and other industries to help enterprises build a foundation for digital transformation.

Copyright Notice: content of this article is contributed spontaneously by Alibaba Cloud real-name registered users. The copyright belongs to the original author. The Alibaba Cloud Developer Community does not own its copyright and does not assume corresponding legal responsibilities. For specific rules, please refer to the "Alibaba Cloud Developer Community User Service Agreement" and the "Alibaba Cloud Developer Community Intellectual Property Protection Guidelines". If you find suspected plagiarism in this community, fill in the infringement complaint form to report it. Once verified, the community will immediately delete the suspected infringing content.