C - How to quickly generate a C extension .md for Python

Original address:

https://blog.52sox.com/shi-yong-cru-he-kuai-su-sheng-cheng-pythonde-ckuo-zhan

---

foreword

I haven't shared technical articles for a long time, mainly because the articles I wrote are too niche, I didn't expect that I would share such an article. Such an article condenses my practical experience in project development over the years.
It has to be said that Python is a very good programming language. Sometimes, for the sake of project code security, some source code will be written in the C API provided by Python, and then compiled into a Python module, so that it can be called directly in Python.

Some tips for compiling

For code written in Python's C API, if quickly generated into corresponding Python extensions, such as under Linux .so or under Windows .pyd .
There are mainly the following methods:

• Using the setuptools module
• Use scikit-build module
• use cmake

In practice these methods are often used together.
For manually writing the setup.py file, specifying the corresponding header file and link library to be imported is not recommended for medium and large projects.

setuptools way

Before that, use the following method to generate a C source file:

pip install pyd-tpl
pyd-tpl hello

This module is a library for writing your own C extension files for quickly generating Python. After the installation is complete, we call the pyd-tpl command to generate a hello.c source file and a setup_hello.py file in the current directory.
We can directly execute the following way to compile and generate a hello module:

python setup_hello.py

This is the first way to generate modules using setuptools. The following two methods are mainly introduced.

scikit-build method

First we run the following command to install the corresponding library:

pip install scikit-build cmake

Since scikit-build depends on cmake, we can install the tool directly using pip.
After the installation is complete, then we write the following code setup.py

from skbuild import setup

CMakeLists.txt in the project directory with the following contents:

cmake_minimum_required(VERSION 3.11.0)
project(hello VERSION 0.1.0)
find_package(PythonExtensions REQUIRED)
add_library("${PROJECT_NAME}" MODULE hello.c)
python_extension_module("${PROJECT_NAME}")
install(TARGETS hello LIBRARY DESTINATION hello)

Here we set the name of our project to hello and look for Python extensions find_package Then add_library the hello.c file through 061dcfd06686da to generate a module named after the project name.
Then you need to pyproject.toml a 061dcfd06686ec file in the project directory, the contents of which are as follows:

[build-system]
requires = ["setuptools", "wheel", "scikit-build", "cmake", "ninja"]

After that, we can run the following command to realize the automatic compilation of the whole process:

python setup.py build

However, the disadvantage of this method is that if the system does not recognize your compiler properly, for example, I am using Visual Studio 2015 under Windows, it is not well recognized. It can only be run under the menu window provided by the Visual Studio 2015 tools.
The process is shown in the figure below:

cmake native way

In cmake version 3.12 start, provides a very easy to use Python_add_library command, details can reference . With this command, we can quickly generate C extensions for Python.
We only need to make the following modifications to the content CMakeLists.txt

cmake_minimum_required(VERSION 3.12.0)
project(hello VERSION 0.1.0)
find_package (Python3 COMPONENTS Development)
include_directories(${Python3_INCLUDE_DIRS})
Python3_add_library(hello MODULE hello.c)

The command syntax is the same as the add_library Here, we are using Python3, so find_package is used as specified in the 061dcfd06687c0 command.
Then create a build directory under Windows, and enter the directory and execute the following command:

mkdir build
cd build
cmake -G "Visual Studio 14 2015" -A x64 ..
cmake --build . --config Release

And for Linux, modify the corresponding command to:

cmake ..
make

Through the cmake tool, we can easily generate C extensions hello.pyd and hello.so

afterword

Indeed, for the generation of Python C extension it is to have 1 cmake ways with setup.py documents incorporated by setup.py file setup function cmdclass custom cmake a compiled class specified parameters.
In fact, in actual project development, there is really no need to make it so complicated. It is not worth the gain to complicate a very simple problem.
I will not introduce it here. Using the above three methods can easily solve the problem of Python's C extension compilation. Among them, the third method is the most recommended.