[Live Preview] Optimizer and Flink CDC + OceanBase Full Incremental Integrated Data Integration Solution

The OceanBase community online technology salon aims to help fans who are concerned about distributed database technology to provide space for technical exchange, sharing and discussion, so that everyone can avoid pitfalls and detours.

1-2 sessions per month, focusing on OceanBase's core technology and peripheral tools, other content you want to hear, welcome to leave a message: [In-depth OceanBase Series] Event Preview + Topic Collection + Winning Announcement

Live Preview

At 19:00 on April 14th (Thursday), the second issue of OceanBase will be broadcast online. It will analyze the optimizer and Flink CDC + OceanBase full-incremental integrated data integration solution in depth. You are welcome to pay attention in advance.

Topic: Optimizer and Flink CDC + OceanBase Full Incremental Integrated Data Integration Solution

Time: April 14th (Thursday) 19:00

Scan the QR code of the poster and enter the group to book a live broadcast

Topic 1: OceanBase Query Optimizer

Lecturer: Wang Guoping (flower name: Xifeng) OceanBase senior technical expert, OceanBase query optimizer leader.

Introduction:

The query optimizer is the core module of the relational database system, the focus and difficulty of the database kernel development, and the "touchstone" for measuring the maturity of the entire database system. It has been more than 40 years since the birth of query optimization theory. In fact, academia and industry have formed a set of relatively complete query optimization frameworks (System-R's Bottom-up dynamic programming optimization framework and Volcano/Cascade's Top-down dynamic programming framework). Planning and optimization framework), but the core problem around query optimization has remained the same - how to use limited system resources to choose a "good" execution plan for the query as much as possible.

This sharing will introduce the engineering implementation of the OceanBase query optimizer and some of our thoughts on the mature optimizer.

1. Introduction to the query optimizer

2. Query rewrite

3. Query optimization

4. Statistics and cost models

5. Program Management (SPM)

6. Thoughts on mature optimizers

Topic 2 : Flink CDC + OceanBase Full Incremental Integrated Data Integration Solution

Lecturer: Wang He (flower name: Chuanfen) OceanBase technical expert.

Introduction:

Flink CDC Connectors is a set of data source connectors of Apache Flink. Users can use Flink CDC to perform full incremental and integrated reading of different data sources, and use Flink SQL to perform stream-batch integrated ETL processing. Currently, support for OceanBase Community Edition has been implemented in the latest release of Flink CDC Connectors 2.2.

This sharing will bring Flink CDC + OceanBase full incremental integrated data integration solution from the following three parts:

1. Traditional CDC and ETL

CDC: DataX + Canal
ETL: CDC + Kafka + Flink
2. Flink CDC + OceanBase

Flink's dynamic tables https://nightlies.apache.org/flink/flink-docs-master/en/docs/dev/table/concepts/dynamic_tables/
Architecture of Flink CDC
Advantages and disadvantages of Flink CDC OceanBase Connector
3. Use of Flink CDC OceanBase Connector

Typical application scenarios of the implementation principle of Flink CDC OceanBase Connector (data synchronization, OB + MySQL table aggregation, etc.)
Hands

Flink CDC Architecture

past review

On March 24th, the content of the first phase of "Introduction to OceanBase" will be broadcast: OceanBase 2022 Roadmap and operation and maintenance actual combat sharing.

Topic 1: OceanBase Open Source Director Ji Junxiang talks about the 2022 Roadmap of the Community Edition

Topic 2: Taking different usage scenarios of Ant Group as an example, analyze how to adjust and optimize high-performance OceanBase

Live Replay: [Introduction to OceanBase Phase 1] OceanBase 2022 Roadmap and Operation and Maintenance Practice

Relevant information:
Flink CDC 2.2 is released, adding OceanBase data source access

【OceanBase series in simple language】Event Preview + Topic Collection + Winning Announcement

For more event information, please visit the "Events" section of the OceanBase open source official website.