如何使用 loguru 接管程序的所有日志输出?

ponponon
English

背景和痛点——日志的要求:输出格式统一

loguru 是多么的优秀就不用我介绍了,我们自己的业务代码,可以轻松的 import loguru 来打印日志,疑惑的一个统一的输出

但是很多第三方库集成的日志模块是标准库中的 logging,其 format 多种多样

我希望可以实现:使用 loguru 接管所有库的 logging,统一使用 loguru 的格式输出

原因?因为方便日志采集和解析存储呀!比如对接 lokislselk 等等。

最佳实践——我理想的日志输出格式:

我希望每条日志输出都长这样

{
    "text": "2022-04-25 18:12:40.179 | ERROR    | __main__:<module>:18 - An error has been caught in function '<module>', process 'MainProcess' (60788), thread 'MainThread' (139849590506112):\nTraceback (most recent call last):\n\n> File \"/home/bot/Desktop/ideaboom/test_logger/loguru_contextualize.py\", line 18, in <module>\n    div(1,0)\n    └ <function div at 0x7f3143edf490>\n\n  File \"/home/bot/Desktop/ideaboom/test_logger/loguru_contextualize.py\", line 8, in div\n    return a/b\n           │ └ 0\n           └ 1\n\nZeroDivisionError: division by zero\n",
    "record": {
        "elapsed": { "repr": "0:00:00.007519", "seconds": 0.007519 },
        "exception": {
            "type": "ZeroDivisionError",
            "value": "division by zero",
            "traceback": true
        },
        "extra": {
            "context_id": "fd77b1d159224d939c1cc0d7a566d09b",
            "message_id": "8ad2c351ef3240998fc10a5015c591c1"
        },
        "file": {
            "name": "loguru_contextualize.py",
            "path": "/home/bot/Desktop/ideaboom/test_logger/loguru_contextualize.py"
        },
        "function": "<module>",
        "level": { "icon": "❌", "name": "DEUBG", "no": 40 },
        "line": 18,
        "message": "An error has been caught in function '<module>', process 'MainProcess' (60788), thread 'MainThread' (139849590506112):",
        "module": "loguru_contextualize",
        "name": "__main__",
        "process": { "id": 60788, "name": "MainProcess" },
        "thread": { "id": 139849590506112, "name": "MainThread" },
        "time": {
            "repr": "2022-04-25 18:12:40.179070+08:00",
            "timestamp": 1650881560.17907
        }
    }
}

对应的代码就是这样:

from loguru import logger
from mark import BASE_DIR
import logging

logger.add(BASE_DIR/'run.log', serialize='json')

我希望 loguru 的日志输出到文件,然后日志收集程序收集日志文件。而不是标准输出,我希望的标准输出是给人类看的,不是给日志收集程序的。

也就是说日志输出两份:

  • 一个输出到 stdoutstderror,可以通过 taildocker-compose logskubectl logs 等等命令直接查看,给人类看(格式不需要统一,反正不统一人类也看的懂,并且这个就是简单的 text,不是 josn 之类的);
  • 另一份输出到 run.log 文件,logtailpromtail 等等日志收集程序就收集 run.log 中的日志记录。(拥有统一的日志格式,并且一行就是一个 json)

以 json 输出日志的优点:

  • 方便采集,不存在因为格式问题导致的采集错误!也不用焦虑堆栈换行而需要使用行首正则匹配表达式等等
  • 方便解析,对接 esmongodb 很方便

缺点呢?

  • 体积膨胀,json 相比直接的 text,体积大了 n 倍

代码实现—— loguru 偷天换日,鱼目混珠

建立一个 loggers.py 文件,内容如下:

from loguru import logger
from mark import BASE_DIR
import logging

logger.add(BASE_DIR/'run.log', serialize='json')


class InterceptHandler(logging.Handler):
    def emit(self, record):
        # Retrieve context where the logging call occurred, this happens to be in the 6th frame upward
        logger_opt = logger.opt(depth=6, exception=record.exc_info)
        logger_opt.log(record.levelno, record.getMessage())


# logging.getLogger("uvicorn").setLevel(10)
# logging.getLogger("uvicorn").handlers = []
# logging.getLogger("uvicorn").addHandler(InterceptHandler())

# logging.getLogger("uvicorn.error").setLevel(10)
# logging.getLogger("uvicorn.error").handlers = []
# logging.getLogger("uvicorn.error").addHandler(InterceptHandler())

# logging.getLogger("fawn.error").setLevel(10)
# logging.getLogger("fawn.error").handlers = []
# logging.getLogger("fawn.error").addHandler(InterceptHandler())

logger_name_list = [name for name in logging.root.manager.loggerDict]
# logger_name_list = [name for name in logging.root.manager.loggerDict if '.' not in name]

for logger_name in logger_name_list:
    logging.getLogger(logger_name).setLevel(10)
    logging.getLogger(logger_name).handlers = []
    if '.' not in logger_name:
        logging.getLogger(logger_name).addHandler(InterceptHandler())


print(logger_name_list)
print(logging.root.manager.loggerDict)

现实中,比如 uvicornpeewee 都有自己的输出格式,如下:

─➤  python api.py
INFO:     Started server process [2382519]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:8000 (Press CTRL+C to quit)

使用上面的代码之后,就变成了:

─➤  uvicorn api:app                                                                                                   1 ↵
2022-04-30 23:09:11.283 | INFO     | uvicorn.server:serve:75 - Started server process [2395776]
2022-04-30 23:09:11.283 | INFO     | uvicorn.lifespan.on:startup:45 - Waiting for application startup.
2022-04-30 23:09:11.283 | INFO     | uvicorn.lifespan.on:startup:59 - Application startup complete.
2022-04-30 23:09:11.284 | INFO     | uvicorn.server:_log_started_message:206 - Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)

解释一下代码

上面这段代码你知道是什么意思吗?

todo

参考文章:
过滤器对象
how can i logging peewee with loguru
How To Override Uvicorn Logger in FastAPI using Loguru

阅读 876

python后端实战经验分享
Python来自荷兰,面向未来

Python 后端工程师可加:企鹅群:537131912

1.3k 声望
34 粉丝
0 条评论

Python 后端工程师可加:企鹅群:537131912

1.3k 声望
34 粉丝
文章目录
宣传栏