mac m1 arm 安装 confluent-kafka 报错解决方案

                    'c.          [e-mail] 
                 ,xNMM.          ----------------- 
               .OMMMMo           OS: macOS 11.6 20G165 arm64 
               OMMM0,            Host: MacBookPro17,1 
     .;loddo:' loolloddol;.      Kernel: 20.6.0 
   cKMMMMMMMMMMNWMMMMMMMMMM0:    Uptime: 3 days, 22 hours, 18 mins 
 .KMMMMMMMMMMMMMMMMMMMMMMMWd.    Packages: 44 (brew) 
 XMMMMMMMMMMMMMMMMMMMMMMMX.      Shell: zsh 5.8 
;MMMMMMMMMMMMMMMMMMMMMMMM:       Resolution: 1440x900, 2560x1440 
:MMMMMMMMMMMMMMMMMMMMMMMM:       DE: Aqua 
.MMMMMMMMMMMMMMMMMMMMMMMMX.      WM: Quartz Compositor 
 kMMMMMMMMMMMMMMMMMMMMMMMMWd.    WM Theme: Blue (Light) 
 .XMMMMMMMMMMMMMMMMMMMMMMMMMMk   Terminal: pycharm 
  .XMMMMMMMMMMMMMMMMMMMMMMMMK.   CPU: Apple M1 
    kMMMMMMMMMMMMMMMMMMMMMMd     GPU: Apple M1 
     ;KMMMMMMMWXXWMMMMMMMk.      Memory: 1616MiB / 8192MiB 
       .cooc,.    .,coo:.
                                                         
                                                         

在 Linux 上没有问题，但是在 Mac 上却不行了，安装 confluent-kafka 会报找不到相关头文件的错误，因为 confluent-kafka 使用了用 C语言编写的 librdkafka 作为基础，而 Mac 糟糕的头文件管理方式让一切变得糟糕。

使用下面的命令安装 confluent-kafka 的时候报错了

pip install confluent-kafka

报错内容如下:

ERROR: Command errored out with exit status 1:
     command: /Library/Frameworks/Python.framework/Versions/3.9/bin/python3.9 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/3r/fkd7zds92j16b43nby2yv8s80000gn/T/pip-install-pura_lpm/confluent-kafka_889dcb8c9d304f88b93b91571870ef8e/setup.py'"'"'; __file__='"'"'/private/var/folders/3r/fkd7zds92j16b43nby2yv8s80000gn/T/pip-install-pura_lpm/confluent-kafka_889dcb8c9d304f88b93b91571870ef8e/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/3r/fkd7zds92j16b43nby2yv8s80000gn/T/pip-record-7jy58dai/install-record.txt --single-version-externally-managed --compile --install-headers /Library/Frameworks/Python.framework/Versions/3.9/include/python3.9/confluent-kafka
         cwd: /private/var/folders/3r/fkd7zds92j16b43nby2yv8s80000gn/T/pip-install-pura_lpm/confluent-kafka_889dcb8c9d304f88b93b91571870ef8e/
...
 In file included from /private/var/folders/3r/fkd7zds92j16b43nby2yv8s80000gn/T/pip-install-pura_lpm/confluent-kafka_889dcb8c9d304f88b93b91571870ef8e/src/confluent_kafka/src/Admin.c:17:
    /private/var/folders/3r/fkd7zds92j16b43nby2yv8s80000gn/T/pip-install-pura_lpm/confluent-kafka_889dcb8c9d304f88b93b91571870ef8e/src/confluent_kafka/src/confluent_kafka.h:23:10: fatal error: 'librdkafka/rdkafka.h' file not found
    #include <librdkafka/rdkafka.h>
             ^~~~~~~~~~~~~~~~~~~~~~
    1 error generated.
    error: command '/usr/bin/gcc' failed with exit code 1
    ----------------------------------------
ERROR: Command errored out with exit status 1: /Library/Frameworks/Python.framework/Versions/3.9/bin/python3.9 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/private/var/folders/3r/fkd7zds92j16b43nby2yv8s80000gn/T/pip-install-pura_lpm/confluent-kafka_889dcb8c9d304f88b93b91571870ef8e/setup.py'"'"'; __file__='"'"'/private/var/folders/3r/fkd7zds92j16b43nby2yv8s80000gn/T/pip-install-pura_lpm/confluent-kafka_889dcb8c9d304f88b93b91571870ef8e/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /private/var/folders/3r/fkd7zds92j16b43nby2yv8s80000gn/T/pip-record-7jy58dai/install-record.txt --single-version-externally-managed --compile --install-headers /Library/Frameworks/Python.framework/Versions/3.9/include/python3.9/confluent-kafka Check the logs for full command output.

解决办法，安装 librdkafka

brew install librdkafka

光着还不够哦，此时还是找不到头文件的，我们还需要让 pip 可以找到 librdkafka 在哪里(Linux 请无视)

Looks like Python is not making use of the homebrew installation location (which should be linked to /usr/local/), try fixing the homebrew package with brew link --overwrite librdkafka and if that doesn't work you can force it in the right direction by:

C_INCLUDE_PATH=/usr/local/Cellar/librdkafka/0.11.0/include LIBRARY_PATH=/usr/local/Cellar/librdkafka/0.11.0/lib pip install confluent_kafka

因此需要设置两个环境变量 C_INCLUDE_PATH 和 LIBRARY_PATH

🚫 我安装的 librdkafka 版本是 1.7.0,记得把下面命令中的版本号替换为你自己的

C_INCLUDE_PATH=/opt/homebrew/Cellar/librdkafka/1.7.0/include LIBRARY_PATH=/opt/homebrew/Cellar/librdkafka/1.7.0/lib pip install confluent_kafka

TIPS:上面是一行命令，千万不要拆开执行，不然无效

有了这两个环境变量，pip 才能正确找到头文件在哪里！

参考文章：
linux gcc头文件搜索路径
Install librdkafka dependency on Mac OSX