The message delivered by nameko is persistent by default.

However, rabbitmq is worried about the throughput of persistent information. Sometimes, non-persistent information is required to speed up.


Just set delivery_mode=NON_PERSISTENT to 0 (NON_PERSISTENT is 0)

 from loguru import logger
import settings
from nameko.standalone.events import event_dispatcher
from nameko.constants import NON_PERSISTENT
import time

config = {
    'AMQP_URI': f'amqp://{settings.RABBITMQ_CONFIG.username}:'
                f'{settings.RABBITMQ_CONFIG.password}@{settings.RABBITMQ_CONFIG.host}:'
                f'{settings.RABBITMQ_CONFIG.port}/{settings.RABBITMQ_CONFIG.vhost}'
}


dispatch = event_dispatcher(config,delivery_mode=NON_PERSISTENT)
for _ in range(100000):
    dispatch(
        'test_publishe',
        'to_publish',
        '1234567890'
    )

The message is not persistent, and the push rate can reach 2.5k/s
图片.png

The above picture is what I pushed to the server through wifi on my macbook, and the delay is as follows:

 PING 192.168.31.245 (192.168.31.245): 56 data bytes
64 bytes from 192.168.31.245: icmp_seq=0 ttl=64 time=4.563 ms
64 bytes from 192.168.31.245: icmp_seq=1 ttl=64 time=4.206 ms
64 bytes from 192.168.31.245: icmp_seq=2 ttl=64 time=3.787 ms
64 bytes from 192.168.31.245: icmp_seq=3 ttl=64 time=3.741 ms
64 bytes from 192.168.31.245: icmp_seq=4 ttl=64 time=4.791 ms
64 bytes from 192.168.31.245: icmp_seq=5 ttl=64 time=4.327 ms
64 bytes from 192.168.31.245: icmp_seq=6 ttl=64 time=3.905 ms
64 bytes from 192.168.31.245: icmp_seq=7 ttl=64 time=4.072 ms
64 bytes from 192.168.31.245: icmp_seq=8 ttl=64 time=4.170 ms
64 bytes from 192.168.31.245: icmp_seq=9 ttl=64 time=4.190 ms
64 bytes from 192.168.31.245: icmp_seq=10 ttl=64 time=10.588 ms
64 bytes from 192.168.31.245: icmp_seq=11 ttl=64 time=4.108 ms
^C
--- 192.168.31.245 ping statistics ---
12 packets transmitted, 12 packets received, 0.0% packet loss
round-trip min/avg/max/stddev = 3.741/4.704/10.588/1.797 ms

What if message persistence is turned on? How fast can it go?

Use delivery_mode=PERSISTENT

图片.png

As you can see, the speed is in the early 200s

The hard disk of the server is SSD Samsung pm981

But I am very strange, is rabbitmq so little throughput?

Then I wrote a producer naked with kombu

 from vine.promises import promise
from kombu import Exchange, Queue
from kombu import Connection
from kombu.messaging import Producer
from kombu.transport.base import Message

from kombu import Exchange, Queue
from loguru import logger
import time


amqp_uri = 'amqp://pon:pon@192.168.31.245:5672//'


def declare_exchange(exchange: Exchange):
    with Connection(amqp_uri) as conn:
        with conn.channel() as channel:
            exchange.declare(channel=channel)


def declare_queue(queue: Queue):
    with Connection(amqp_uri) as conn:
        with conn.channel() as channel:
            queue.declare(channel=channel)


imdb_exchange = Exchange('imdb', type='fanout')
declare_exchange(exchange=imdb_exchange)

imdb_queue = Queue('imdb_refresh', imdb_exchange,
                   routing_key='to_refresh', durable=True)
declare_queue(queue=imdb_queue)


with Connection(amqp_uri) as conn:
    with conn.channel() as channel:
        started_at = time.time()
        message = Message(channel=channel, body='123456789')

        producer = Producer(
            channel,
            exchange=imdb_exchange
        )
        for _ in range(1000000):
            res = producer.publish(
                body=message.body,
                routing_key='to_refresh',
                headers=message.headers
            )
        ended_at = time.time()
        logger.debug(f'pay time {ended_at-started_at} s')
        # logger.debug(res)

As shown in the figure:

图片.png

The gap is a bit big, it's all 3w/s!


universe_king
3.4k 声望678 粉丝