What is the difference between kombu's serializer choosing json and yaml?

foreword

When kombu delivers a message, it supports multiple serialization methods:

• json
• yaml
• pickle

Today's topic is to see what the message will look like after a dict is serialized by these three serialization methods (subject to the appearance of the rabbitmq management interface)

experiment

First prepare a dict, the type of value has str, int, datetime, sub dict and other types

 data = {
    'name': 'jike',
    'age': 18,
    'birthday': get_utc_now_timestamp(),
    'score': {
        'math': 100,
        'science': 99.5,
        'english': 59
    }
}

Let's first take a look at the result of not specifying serializer:

 with Connection(amqp_uri) as conn:
    with conn.channel() as channel:
        started_at = time.time()
        message = Message(channel=channel, body=data)

        producer = Producer(
            channel,
            exchange=imdb_exchange
        )

        res = producer.publish(
            body=message.body,
            routing_key='to_refresh',
            headers=message.headers
        )
        ended_at = time.time()
        logger.debug(f'pay time {ended_at-started_at} s')

It can be seen that because the body of the message is a dict, kombu chooses to serialize according to json even if the serializer is missing

How to come to the conclusion of "serialization according to json"? Because looking at the "content_type" attribute in the message header: content_type:application/json

What if we choose json?

 with Connection(amqp_uri) as conn:
    with conn.channel() as channel:
        started_at = time.time()
        message = Message(channel=channel, body=data)

        producer = Producer(
            channel,
            exchange=imdb_exchange
        )

        res = producer.publish(
            body=message.body,
            routing_key='to_refresh',
            headers=message.headers,
            serializer='json'
        )
        ended_at = time.time()
        logger.debug(f'pay time {ended_at-started_at} s')

As you can see, there is no difference from the above, both are content_type: application/json

What if we choose yaml?

 with Connection(amqp_uri) as conn:
    with conn.channel() as channel:
        started_at = time.time()
        message = Message(channel=channel, body=data)

        producer = Producer(
            channel,
            exchange=imdb_exchange
        )

        res = producer.publish(
            body=message.body,
            routing_key='to_refresh',
            headers=message.headers,
            serializer='yaml'
        )
        ended_at = time.time()
        logger.debug(f'pay time {ended_at-started_at} s')

As you can see, the dict object we passed becomes yaml text

Let's look at pickle again:

As you can see, at this point, we can't see the body, because pickle is a binary serialization method

Full code:

 from kombu import Exchange, Queue
from kombu import Connection
from kombu.messaging import Producer
from kombu.transport.base import Message

from kombu import Exchange, Queue
from loguru import logger
import time
from datetime import datetime, timedelta, timezone


def get_min_utc_timestamp() -> datetime:
    return (datetime(year=1970, month=1, day=1) + timedelta(seconds=1)).replace(tzinfo=timezone.utc)


def get_utc_now_timestamp() -> datetime:
    """ https://blog.csdn.net/ball4022/article/details/101670024 """
    return datetime.utcnow().replace(tzinfo=timezone.utc)


amqp_uri = 'amqp://pon:pon@192.168.31.245:5672//'


def declare_exchange(exchange: Exchange):
    with Connection(amqp_uri) as conn:
        with conn.channel() as channel:
            exchange.declare(channel=channel)


def declare_queue(queue: Queue):
    with Connection(amqp_uri) as conn:
        with conn.channel() as channel:
            queue.declare(channel=channel)


imdb_exchange = Exchange('imdb', type='fanout')
declare_exchange(exchange=imdb_exchange)

imdb_queue = Queue('imdb_refresh', imdb_exchange,
                   routing_key='to_refresh', durable=True)
declare_queue(queue=imdb_queue)


data = {
    'name': 'jike',
    'age': 18,
    'birthday': get_utc_now_timestamp(),
    'score': {
        'math': 100,
        'science': 99.5,
        'english': 59
    }
}

with Connection(amqp_uri) as conn:
    with conn.channel() as channel:
        started_at = time.time()
        message = Message(channel=channel, body=data)

        producer = Producer(
            channel,
            exchange=imdb_exchange
        )

        res = producer.publish(
            body=message.body,
            routing_key='to_refresh',
            headers=message.headers,
            serializer='pickle'
        )
        ended_at = time.time()
        logger.debug(f'pay time {ended_at-started_at} s')

References:
kombu doc: serialization