Data intelligent construction management platform Dataphin V2.9.4.3 released

Introduction to Dataphin released version V2.9.4.3 to upgrade multiple product capabilities. This version has optimized and improved product functions and user experience, aiming to provide users with more complete product capabilities and experience to accelerate the enterprise Data center construction process.

-For more information about digital intelligence transformation and data center content, please join Cloud Data Center Exchange Group-Digital Intelligence Club and follow the official WeChat official at the end of the article to join )

Cloud Data Center official website 160f7cd007bcd0 https://dp.alibaba.com/index

---

1 Product introduction

Dataphin is a productized output of Alibaba Group’s OneData data governance methodology based on internal practices. It provides one-stop data acquisition, construction, management, and use of the full life cycle management capabilities to help companies significantly improve the level of data governance, and build reliable quality and consumption Convenient, safe and economical enterprise-level data center. Dataphin provides a variety of computing engine support and expandable open capabilities to adapt to the platform technology architecture and personalized requirements of all walks of life.

2 Version overview

In June 2021, Dataphin released the V2.9.4.3 version, which upgraded a number of product capabilities.

• In terms of platform capabilities, expand computing engine richness and OpenAPI coverage
• The data integration module expands the supported MySQL data source version and the coverage of one-click table creation to improve configuration efficiency
• In terms of monitoring capabilities, optimize the configuration of alarm receiving rules to improve flexibility and adapt to more monitoring scenarios
• Asset center, optimize the logic table preview and sensitive field identification rules, and improve the asset link
• Data service expands API paging query capabilities to expand query scope and improve service response efficiency and link stability

This version has been optimized and improved in terms of product functions and user experience, and aims to provide users with more complete product capabilities and experience, so as to accelerate the construction of enterprise data centers.

3 Detailed explanation of key features of the new version

Feature 1: The calculation engine adds support for CDH6

New adaptations for the CDH6 computing engine are added to improve the multi-engine compatibility; as of the current version, the types of computing engines supported by Dataphin include: MaxCompute, CDH5, CDH6, and EMR.

Feature 2: MySQL data source supports 8.x version

Currently, 8.0 is the more mainstream and widely used MySQL database version on the market. Dataphin has already supported MySQL 5.6 and 5.7 data sources and added support for MySQL 8.0 version, which can be configured based on this data source in modules such as data synchronization and data services to improve business data coverage.

Feature 3: Data integration supports one-click table creation in the Oracle target database

Supports one-click creation of data tables in the target Oracle target database to simplify the configuration process and improve the efficiency of data synchronization configuration. As of the current version, the one-click table creation function covers a total of 4 target data sources: MaxCompute, Oracle, Hive, and AnalyticDB for PostgreSQL.

Feature 4: Task operation monitoring and quality monitoring support assigning different alarms and receiving methods to different recipients

Before the upgrade, for all selected recipients, the same alarm receiving method needs to be configured. After upgrading to this version, for different recipient types, you can specify different receiving methods to achieve differentiated alarms based on actual conditions. For example, the person in charge of the task needs to understand the operation overview of the task he is responsible for, but does not need to deal with the abnormal situation immediately, and can choose SMS alert; the person in charge of duty needs to discover the abnormality in time and deal with it, and can choose to call as a strong reminder; the project manager needs to regularly Statistics alarm overview, you can select email alarms to facilitate recording and statistics.

Feature 5: The asset map has a new logical table data preview function, and asset security supports manual triggering of sensitive field identification

The new logic table preview function is added to directly display the sampled data records for fields with permissions. If the field is set with desensitization rules, only the data after desensitization will be displayed; for fields with no permissions, the word "no permission" will be displayed. Provide a jump link for quick application. Combined with this function, Dataphin has perfected the entire link of the logic table from R&D to asset precipitation to consumption preview, enhancing the modeling experience.

By default, after the asset security module is configured with sensitive data identification rules, the next day will start to scan every day. This time, on the basis of daily scheduled scans, users are newly added to manually trigger the operation of sensitive data identification tasks, so that new rules take effect immediately, and records in temporary change scenarios are updated in time to improve the coverage of sensitive data identification scenarios.

Feature 6: The data service supports API paging queries based on Impala data sources to expand the scope of queries and improve query stability

In the historical version, considering the query performance, the API single query created based on the Impala data source only supports the return of a maximum of 1000 results, which cannot meet the large data volume query scenario and affects the use of downstream businesses. This time, for the API created based on the Impala data source, it provides paging query capabilities, supports setting paging conditions through limit or offset statements to ensure service connection stability and corresponding efficiency, and supports large data volume query scenarios.

4 Summary and outlook

In the V2.9.4.3 version released this time, Dataphin has iteratively upgraded the functions of computing engine, data source, data integration, monitoring alarm, and data service; in the next version, we will focus on supporting FusionInsight computing engine adaptation, data Functions such as extraction upgrade, OpenAPI expansion, operation and maintenance data supplement capability improvement, data service multi-project, etc., so stay tuned!

Related products: Intelligent data construction and management Dataphin

---

Data center is the only way for enterprises to achieve digital intelligence. Alibaba believes that data center is a combination of methodology, tools, and organization, which is "fast", "quasi", "full", "unified", and "passed". Smart big data system.

Currently by Ali cloud external output range of solutions, including common data desk solution , retail sales data desk solution , financial data desk solution , Internet data desk solution , Subdivision scenarios such as and other subdivision scenarios for government data middle-office solutions.

Among them, the Alibaba Cloud Data Center product matrix is based on Dataphin and the Quick series is used as a business scenario cut-in, including:

• -Dataphin, a one-stop, intelligent data construction and management platform ;
• -Quick BI, intelligent decision-making anytime, anywhere ;
• -Quick Audience, comprehensive insight, global marketing, intelligent growth ;
• -Quick A+, a one-stop data operation platform cross-multi-terminal global application experience analysis and insight;
• -Quick Stock, an intelligent goods operation platform ;
• -Quick Decision, an intelligent decision platform ;

official site:

Data Zhongtai official website https://dp.alibaba.com

Dingding Communication Group and WeChat Official Account

Copyright Statement: content of this article is contributed spontaneously by Alibaba Cloud real-name registered users, and the copyright belongs to the original author. The Alibaba Cloud Developer Community does not own the copyright, and does not bear the corresponding legal responsibility. For specific rules, please refer to the "Alibaba Cloud Developer Community User Service Agreement" and the "Alibaba Cloud Developer Community Intellectual Property Protection Guidelines". If you find suspected plagiarism in this community, fill in the infringement complaint form to report it. Once verified, the community will immediately delete the suspected infringing content.