Golang implements RabbitMQ's dead letter queue

Before reading this article, you should already understand some concepts of RabbitMQ, such as queues, exchanges and so on.

Dead letter concept

In layman's terms, messages that cannot be consumed normally are called dead letters. We put it on the dead letter queue and handle this part of the "exception" message separately.

When a message meets one of the following conditions, it is called a dead letter.

• The message is rejected and not put back on the queue (messages are rejected using the basic.reject / basic.nack methods, and the parameter requeue = false for both methods)
• Message TTL expired
• queue reaches maximum length

application

Application scenario: When consumers cannot consume messages normally and the message is abnormal, in order to ensure that data is not lost, the abnormal message is set as a dead letter and placed in the dead letter queue. Messages in the dead letter queue will start a separate consumer program for special processing.

Architecture diagram:

Follow the architecture diagram below to implement the code.

producer

A producer generally only needs to do two things, one is to create a link, and the other is to send a message.

The queues, exchanges, and routing-keys involved in RabbitMQ all need to be created in code. These operations can be created by both producers and consumers. For a discussion of who will create, see RabbitMq: Who will create queues and exchanges? this article.

In this article, queues, switches, and routing-keys are implemented on the producer side. So producers need to do a total of these things.

1. Create a connection
2. Set up queues (queues, exchanges, bindings)
3. Set up dead letter queues (queues, exchanges, bindings)
4. release the news

Create a connection

Use the strreadway/amqp package to establish a connection with RabbitMQ.

func main() {
    mq := util.NewRabbitMQ()
    defer mq.Close()
    mqCh := mq.Channel

    ……
}

……

// util.NewRabbitMQ()

func NewRabbitMQ() *RabbitMQ {
    conn, err := amqp.Dial(constant.MqUrl)
    FailOnError(err, "Failed to connect to RabbitMQ")

    ch, err := conn.Channel()
    FailOnError(err, "Failed to open a channel")

    return &RabbitMQ{
        Conn: conn,
        Channel: ch,
    }
}

Set up queues (queues, exchanges, bindings)

The core operation is to set the queue stage.

Declare a normal queue, specify a dead-letter switch, and specify a dead-letter routing-key. subsequent dead letter queue is created, it will be bound to the dead letter switch and the specified dead letter routing-key .

var err error
_, err = mqCh.QueueDeclare(constant.NormalQueue, true, false, false, false, amqp.Table{
    "x-message-ttl": 5000, // 消息过期时间,毫秒
    "x-dead-letter-exchange": constant.DeadExchange, // 指定死信交换机
    "x-dead-letter-routing-key": constant.DeadRoutingKey, // 指定死信routing-key
})
util.FailOnError(err, "创建normal队列失败")

declare switch

err = mqCh.ExchangeDeclare(constant.NormalExchange, amqp.ExchangeDirect, true, false, false, false, nil)
util.FailOnError(err, "创建normal交换机失败")

At present, common queues and exchanges have been created, but they exist independently and are not associated.

The queue, routing-key, and switch are bound together by QueueBind.

err = mqCh.QueueBind(constant.NormalQueue, constant.NormalRoutingKey, constant.NormalExchange, false, nil)
util.FailOnError(err, "normal：队列、交换机、routing-key 绑定失败")

Set up dead letter queues (queues, exchanges, bindings)

The same dead letter queue, also need to create queues, create exchanges and bindings.

// 声明死信队列
// args 为 nil。切记不要给死信队列设置消息过期时间,否则失效的消息进入死信队列后会再次过期。
_, err = mqCh.QueueDeclare(constant.DeadQueue, true, false, false, false, nil)
util.FailOnError(err, "创建dead队列失败")

// 声明交换机
err = mqCh.ExchangeDeclare(constant.DeadExchange, amqp.ExchangeDirect, true, false, false, false, nil)
util.FailOnError(err, "创建dead队列失败")

// 队列绑定（将队列、routing-key、交换机三者绑定到一起）
err = mqCh.QueueBind(constant.DeadQueue, constant.DeadRoutingKey, constant.DeadExchange, false, nil)
util.FailOnError(err, "dead：队列、交换机、routing-key 绑定失败")

When the dead letter queue is established, the common queue x-dead-letter-exchange and x-dead-letter-routing-key , and the dead letter queue is connected to the common queue.

release the news

message := "msg" + strconv.Itoa(int(time.Now().Unix()))
fmt.Println(message)

// 发布消息
err = mqCh.Publish(constant.NormalExchange, constant.NormalRoutingKey, false, false, amqp.Publishing{
    ContentType: "text/plain",
    Body:        []byte(message),
})
util.FailOnError(err, "消息发布失败")

Producer complete code

package main

import (
    "fmt"
    "github.com/streadway/amqp"
    "learn_gin/go/rabbitmq/deadletter/constant"
    "learn_gin/go/rabbitmq/deadletter/util"
    "strconv"
    "time"
)

func main() {
    // # ========== 1.创建连接 ==========
    mq := util.NewRabbitMQ()
    defer mq.Close()
    mqCh := mq.Channel

    // # ========== 2.设置队列（队列、交换机、绑定） ==========
    // 声明队列
    var err error
    _, err = mqCh.QueueDeclare(constant.NormalQueue, true, false, false, false, amqp.Table{
        "x-message-ttl": 5000, // 消息过期时间,毫秒
        "x-dead-letter-exchange": constant.DeadExchange, // 指定死信交换机
        "x-dead-letter-routing-key": constant.DeadRoutingKey, // 指定死信routing-key
    })
    util.FailOnError(err, "创建normal队列失败")

    // 声明交换机
    err = mqCh.ExchangeDeclare(constant.NormalExchange, amqp.ExchangeDirect, true, false, false, false, nil)
    util.FailOnError(err, "创建normal交换机失败")

    // 队列绑定（将队列、routing-key、交换机三者绑定到一起）
    err = mqCh.QueueBind(constant.NormalQueue, constant.NormalRoutingKey, constant.NormalExchange, false, nil)
    util.FailOnError(err, "normal：队列、交换机、routing-key 绑定失败")

    // # ========== 3.设置死信队列（队列、交换机、绑定） ==========
    // 声明死信队列
    // args 为 nil。切记不要给死信队列设置消息过期时间,否则失效的消息进入死信队列后会再次过期。
    _, err = mqCh.QueueDeclare(constant.DeadQueue, true, false, false, false, nil)
    util.FailOnError(err, "创建dead队列失败")

    // 声明交换机
    err = mqCh.ExchangeDeclare(constant.DeadExchange, amqp.ExchangeDirect, true, false, false, false, nil)
    util.FailOnError(err, "创建dead队列失败")

    // 队列绑定（将队列、routing-key、交换机三者绑定到一起）
    err = mqCh.QueueBind(constant.DeadQueue, constant.DeadRoutingKey, constant.DeadExchange, false, nil)
    util.FailOnError(err, "dead：队列、交换机、routing-key 绑定失败")

    // # ========== 4.发布消息 ==========
    message := "msg" + strconv.Itoa(int(time.Now().Unix()))
    fmt.Println(message)

    // 发布消息
    err = mqCh.Publish(constant.NormalExchange, constant.NormalRoutingKey, false, false, amqp.Publishing{
        ContentType: "text/plain",
        Body:        []byte(message),
    })
    util.FailOnError(err, "消息发布失败")
}

consumer

Since queues and switches are created by producers, consumers only need to do two things, one is to establish a connection, and the other is to consume messages.

For this reason, the consumer starts later than the producer, and it can be guaranteed that the queue exists when the consumer is consumed.

package main

import (
    "learn_gin/go/rabbitmq/deadletter/constant"
    "learn_gin/go/rabbitmq/deadletter/util"
    "log"
)

func main() {
    // # ========== 1.创建连接 ==========
    mq := util.NewRabbitMQ()
    defer mq.Close()
    mqCh := mq.Channel

    // # ========== 2.消费消息 ==========
    msgsCh, err := mqCh.Consume(constant.NormalQueue, "", false, false, false, false, nil)
    util.FailOnError(err, "消费normal队列失败")

    forever := make(chan bool)
    go func() {
        for d := range msgsCh {
            // 要实现的逻辑
            log.Printf("接收的消息: %s", d.Body)

            // 手动应答
            d.Ack(false)
            //d.Reject(true)
        }
    }()
    log.Printf("[*] Waiting for message, To exit press CTRL+C")
    <-forever
}

dead letter consumer

Dead letter queues and switches are also created by producers, and dead letter consumers only need to do two things, establish connections and consume messages.

package main

import (
    "learn_gin/go/rabbitmq/deadletter/constant"
    "learn_gin/go/rabbitmq/deadletter/util"
    "log"
)

func main() {
    // # ========== 1.创建连接 ==========
    mq := util.NewRabbitMQ()
    defer mq.Close()
    mqCh := mq.Channel

    // # ========== 2.消费死信消息 ==========
    msgsCh, err := mqCh.Consume(constant.DeadQueue, "", false, false, false, false, nil)
    util.FailOnError(err, "消费dead队列失败")

    forever := make(chan bool)
    go func() {
        for d := range msgsCh {
            // 要实现的逻辑
            log.Printf("接收的消息: %s", d.Body)

            // 手动应答
            d.Ack(false)
            //d.Reject(true)
        }
    }()
    log.Printf("[*] Waiting for message, To exit press CTRL+C")
    <-forever
}

Source code Mr-houzi/go-demo

end!

Personal blog synchronization article Golang implements RabbitMQ's dead letter queue