MySQL source code reading-macOS CLion compile and debug MySQL 5.7

For macOS + VSCode version see here

Environmental information

• macOS Big Sur 11.5.2
• CLion 2021.2
• MySQL 5.7.35
• CMake 3.21.1
• openssl 1.1

Download source code

Download the source code that boost version from the official website

Download link: https://dev.mysql.com/get/Downloads/MySQL-5.7/mysql-boost-5.7.35.tar.gz

You can also GitHub and switch to the designated TAG or branch.

Patch source code

If MySQL <= 8.0.21 , you need to execute the following script Patch source code:

mv VERSION MYSQL_VERSION
sed -i '' 's|${CMAKE_SOURCE_DIR}/VERSION|${CMAKE_SOURCE_DIR}/MYSQL_VERSION|g' cmake/mysql_version.cmake

For specific reasons, please refer to the article: MySQL source code-problem expanded from macro MYSQL_VERSION_MAJOR

Configure CMake

The CMake options configuration is as follows:

-DWITH_DEBUG=1
-DDOWNLOAD_BOOST=1
-DDOWNLOAD_BOOST_TIMEOUT=60000
-DWITH_BOOST=boost
-DCMAKE_INSTALL_PREFIX=build_out
-DMYSQL_DATADIR=build_out/data
-DSYSCONFDIR=build_out/etc
-DMYSQL_TCP_PORT=3307
-DMYSQL_UNIX_ADDR=mysql-debug.sock

Explain the above parameters:

-DWITH_DEBUG=1                     # 开启DEBUG模式
-DDOWNLOAD_BOOST=1                 # boost不存在时自动下载
-DDOWNLOAD_BOOST_TIMEOUT=60000     # 下载boost的超时时间
-DWITH_BOOST=boost                 # boost目录，不存在时会自动下载到该目录
-DCMAKE_INSTALL_PREFIX=build_out   # MySQL安装目录，可在启动时指定`--basedir`覆盖
-DMYSQL_DATADIR=build_out/data     # MySQL数据目录，可在启动时指定`--datadir`覆盖
-DSYSCONFDIR=build_out/etc         # `my.cnf`默认目录，可在启动时指定`--defaults-file=file_name`覆盖
-DMYSQL_TCP_PORT=3307              # 如果本机已安装过MySQL，避免冲突换个别的
-DMYSQL_UNIX_ADDR=mysql-debug.sock # 默认/tmp/mysql.sock，避免冲突，此处相对与`--datadir`目录会自动创建

All optional parameters: MySQL Source-Configuration Options

Manually create the directory in the configuration:

# 为何加`cmake-build-debug`前缀，因为`CLion`中`CMake`的`Build directory`就是`cmake-build-debug`，可自行修改
mkdir -p cmake-build-debug/build_out cmake-build-debug/build_out/data cmake-build-debug/build_out/etc

Run CMake

Method 1: Tools > CMake > Reset Cache and Reload Project

Method two: View > Tool Windows > CMake > Reset Cache and Reload Project

After execution, the CMake window outputs the following content:

/Applications/CLion.app/Contents/bin/cmake/mac/bin/cmake -DCMAKE_BUILD_TYPE=Debug -DWITH_DEBUG=1 -DDOWNLOAD_BOOST=1 -DDOWNLOAD_BOOST_TIMEOUT=60000 -DWITH_BOOST=boost -DCMAKE_INSTALL_PREFIX=build_out -DMYSQL_DATADIR=build_out/data -DSYSCONFDIR=build_out/etc -DMYSQL_TCP_PORT=3307 -DMYSQL_UNIX_ADDR=mysql-debug.sock -DCMAKE_DEPENDS_USE_COMPILER=FALSE -G "CodeBlocks - Unix Makefiles" /path/to/mysql-5.7.35

······
-- Configuring done
-- Generating done
-- Build files have been written to: /path/to/mysql-5.7.35/cmake-build-debug

Compile mysqld

Click CLion upper right window Build button or shortcut keys ⌘+F9 :

Compiling for the first time will be slower. After the compilation is completed, the following content will be output:

====================[ Build | mysqld | Debug ]==================================
/Applications/CLion.app/Contents/bin/cmake/mac/bin/cmake --build /path/to/mysql-5.7.35/cmake-build-debug --target mysqld -- -j 3
[  0%] Built target regex
··· 中间日志省略 ···
[100%] Built target mysqld

Build finished

Configure my.cnf

If you need other customized configuration, you can add/modify my.cnf

In CMake configuration -DSYSCONFDIR=build_out/etc directory, create my.cnf files, and edit:

[mysqld]
innodb_file_per_table = 1

Initialize mysqld

Click CLion upper right corner of the Edit Configurations... window:

Find mysqld and configure Program arguments :

Program arguments configuration content:

--initialize-insecure

Click CLion upper right window Run icon or shortcut ⌃+R , initialize:

After initialization, the output is as follows:

/path/to/mysql-5.7.35/cmake-build-debug/sql/mysqld --initialize-insecure
2021-08-25T05:35:14.403520Z 0 [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation for more details).
2021-08-25T05:35:14.403970Z 0 [ERROR] Can't find error-message file '/path/to/mysql-5.7.35/cmake-build-debug/build_out/share/errmsg.sys'. Check error-message file location and 'lc-messages-dir' configuration directive.
2021-08-25T05:35:14.405823Z 0 [Warning] Setting lower_case_table_names=2 because file system for /path/to/mysql-5.7.35/cmake-build-debug/build_out/data/ is case insensitive
2021-08-25T05:35:14.658572Z 0 [Warning] InnoDB: New log files created, LSN=45790
2021-08-25T05:35:15.068077Z 0 [Warning] InnoDB: Creating foreign key constraint system tables.
2021-08-25T05:35:15.214272Z 0 [Warning] No existing UUID has been found, so we assume that this is the first time that this server has been started. Generating a new UUID: 398f5a88-0566-11ec-9fc8-c1b33e1edaea.
2021-08-25T05:35:15.218653Z 0 [Warning] Gtid table is not ready to be used. Table 'mysql.gtid_executed' cannot be opened.
2021-08-25T05:35:16.089766Z 0 [Warning] 
2021-08-25T05:35:16.089804Z 0 [Warning] 
2021-08-25T05:35:16.090612Z 0 [Warning] CA certificate ca.pem is self signed.
2021-08-25T05:35:16.385026Z 1 [Warning] root@localhost is created with an empty password ! Please consider switching off the --initialize-insecure option.

At this time, MySQL has automatically created a root@localhost account, and the password is empty!

Then Program arguments configuration --initialize-insecure delete ! ! !

Because the initialization only needs to be executed once, the configuration here will be used for subsequent execution or Debug, and an error will be reported if it is not deleted.

Start DEBUG

Click CLion upper right window Debug icon or shortcut ⌃+D , start Debug:

Connect to the MySQL server through an existing client, such as:

mysql -uroot -P3307 -h127.0.0.1

You can also connect via Navicat:

Then enter a piece of SQL on the client, CLion Debug window will prompt:

common problem

Warning level error, can be ignored

CMake Error at cmake/boost.cmake

wrong description

This error will be reported when cmake

CMake Error at cmake/boost.cmake:88 (MESSAGE):
  You can download it with -DDOWNLOAD_BOOST=1 -DWITH_BOOST=<directory>

  This CMake script will look for boost in <directory>.  If it is not there,
  it will download and unpack it (in that directory) for you.

  If you are inside a firewall, you may need to use an http proxy:

  export http_proxy=http://example.com:80

Call Stack (most recent call first):
  cmake/boost.cmake:245 (COULD_NOT_FIND_BOOST)
  CMakeLists.txt:536 (INCLUDE)

To compile MySQL source code, CMake needs to configure the -DWITH_BOOST=<directory> option.

Confirm whether the source code carries boost

There are two ways to boost is carried in the source code:

• Distinguish by download URL or source file name

https://cdn.mysql.com/Downloads/MySQL-5.7/mysql-5.7.35.tar.gz
https://cdn.mysql.com/Downloads/MySQL-5.7/mysql-boost-5.7.35.tar.gz

• Is there a boost directory in the source code

The source code downloaded from GitHub does not contain boost

Source installation with boost

cmake . -DWITH_BOOST=boost

Source installation without boost

cmake . \
-DDOWNLOAD_BOOST=1 \
-DWITH_BOOST=<directory>

This is the recommended build method for MySQL. CMake will <directory> that meets the version requirements under boost . If it does not exist, it will download and unzip it to this directory.

Example output after executing the build:

...
-- MySQL 5.7.35
-- Packaging as: mysql-5.7.35-osx10.16-x86_64
-- DTRACE is enabled
-- Downloading boost_1_59_0.tar.gz to /path/to/mysql-server-mysql-5.7.35/boost
-- cd /path/to/mysql-server-mysql-5.7.35/boost; tar xfz /path/to/mysql-server-mysql-5.7.35/boost/boost_1_59_0.tar.gz
-- Found /path/to/mysql-server-mysql-5.7.35/boost/boost_1_59_0/boost/version.hpp
-- BOOST_VERSION_NUMBER is #define BOOST_VERSION 105900
-- BOOST_INCLUDE_DIR /path/to/mysql-server-mysql-5.7.35/boost/boost_1_59_0
...

Or manually download and unzip to a directory, and then specify the boost directory WITH_BOOST

wget https://nchc.dl.sourceforge.net/project/boost/boost/1.59.0/boost_1_59_0.tar.gz
tar -zxf boost_1_59_0.tar.gz
cmake . \
-DWITH_BOOST=<directory>

Warning: define bzero please_use_memset_rather_than_bzero

During the build process, the following warning may be reported

[ 31%] Building C object storage/myisammrg/CMakeFiles/myisammrg.dir/myrg_records.c.o
In file included from /path/to/mysql-5.7.35/storage/myisam/mi_page.c:25:
In file included from /path/to/mysql-5.7.35/storage/myisam/myisamdef.h:26:
In file included from /path/to/mysql-5.7.35/include/myisam.h:34:
/path/to/mysql-5.7.35/include/m_string.h:32:9: warning: 'bzero' macro redefined [-Wmacro-redefined]
#define bzero please_use_memset_rather_than_bzero
        ^
/Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/secure/_strings.h:52:9: note: previous definition is here
#define bzero(dest, ...) \
        ^
In file included from /path/to/mysql-5.7.35/storage/myisammrg/myrg_records.c:24:
In file included from /path/to/mysql-5.7.35/storage/myisammrg/myrg_def.h:25:
In file included from /path/to/mysql-5.7.35/storage/myisammrg/../myisam/myisamdef.h:26:
In file included from /path/to/mysql-5.7.35/include/myisam.h:34:
/path/to/mysql-5.7.35/include/m_string.h:32:9: warning: 'bzero' macro redefined [-Wmacro-redefined]
#define bzero please_use_memset_rather_than_bzero
        ^
/Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/secure/_strings.h:52:9: note: previous definition is here
#define bzero(dest, ...) \
        ^
1 warning generated.

MacOSX11.3.sdk when defining bzero :

/Library/Developer/CommandLineTools/SDKs/MacOSX11.3.sdk/usr/include/secure/_strings.h:52:9: note: previous definition is here

#define bzero(dest, ...) \
        __builtin___memset_chk (dest, 0, __VA_ARGS__, __darwin_obsz0 (dest))
#endif

This means that the macro bzero has been defined in the file introduced by mysql

/path/to/mysql-5.7.35/include/m_string.h:32:9: warning: 'bzero' macro redefined [-Wmacro-redefined]

#define bzero please_use_memset_rather_than_bzero

mysql is not a function code. Instead, it prompts that the memset function should be used instead of the bzero function. This means that the MySQL developer will need to modify the code later. Here is a reminder with redefined, which does not affect the actual operation. There is no such prompt in MySQL 8.0.

Therefore, we can ignore such warnings.

expanded from macro MYSQL_VERSION_MAJOR

Refer to MySQL source code-problem expanded from macro MYSQL_VERSION_MAJOR

Compile mysql and other tools

Before that, we only compiled the mysqld server software. But when we need mysql client, or mysqldump , we can also choose to configure other items and click Build .

Reference

• 《MySQL Internals Manual》Building MySQL Server with CMake
• 《MySQL 5.7 Reference Manual》Installing MySQL from Source
• 《MySQL 5.7 Reference Manual》MySQL Source-Configuration Options
• MySQL source code reading-problem expanded from macro MYSQL_VERSION_MAJOR

---

Thank you for reading, think the content is good, please like it 😆
Original address: https://shockerli.net/post/mysql-source-macos-clion-debug-5-7/