message
Mandatory and immediate are the two parameters in the message sending method. They both have the function of returning the message to the producer when the destination is unreachable during the message delivery process. The backup exchange provided by RabbitMQ can store messages that cannot be routed by the exchange without returning to the client.
mandatory parameter
When the mandatory parameter is set to true, the exchange cannot find a qualified queue based on its own type and routing key, then RabbitMQ will return the message to the producer. When the mandatory parameter is set to false, the message is directly discarded.
So how does the producer get the message that has not been routed to the appropriate queue correctly? At this time, it can be achieved by registering the message to return to the listener:
func (ch *Channel) NotifyReturn(c chan Return) chan Return
immediate parameter
When the immediate parameter is set to true, if the exchange finds that there are no consumers on the queue when routing the message to the queue, the message will not be stored in the queue. When all queues matching the routing key have no consumers, the message will be returned to the producer.
In summary, the mandatory parameter tells the server to route the message to at least one queue, otherwise it will return the message to the producer. The immediate parameter tells the server that if there is a consumer on the queue associated with the message, it will be consumed immediately; if there are no consumers on all matching queues, the message will be directly returned to the production for viewing, instead of storing the message in the queue and the consumer .
backup switch
If the producer does not set the mandatory parameter when sending the message, the message will be lost if it is not routed. If the mandatory parameter is set, then the programming logic of ReturnListener needs to be added, and the code of the producer will become complicated. If you do not want to complicate the programming logic of the producer and do not want to lose messages, you can use a backup switch so that unrouted messages can be stored in RabbitMQ, and then processed when needed.
can be implemented by setting the alternate-exchange option in the args parameter when declaring the exchange, or it can be implemented by strategy. If the two are used at the same time, the former has a higher priority and will override the Policy setting.
The backup switch is actually not much different from the ordinary switch. It should be noted that the routing key when the message is resent to the backup switch is the same as the routing key sent from the producer.
For the backup switch, the following special cases are summarized:
- If the set backup switch does not exist, neither the client nor the RabbitMQ server will be abnormal, and the message will be lost at this time.
- If the backup switch is not bound to any queue, there will be no exceptions on the client and RabbitMQ server, and the message will be lost at this time.
- If the backup switch does not have any matching queues, there will be no exceptions on the client and RabbitMQ server, and the message will be lost at this time.
- If the backup switch is used with the mandatory parameter, then the mandatory parameter is invalid.
Expiration time (TTL)
Set the TTL of the message
There are currently two ways to set the TTL of a message. first method is to set the queue properties, all messages in the queue have the same expiration time. The second method is to set the message itself separately, and the TTL of each message can be different. If the two methods are used together, the TTL of the message is based on the smaller value between the two. message's survival time in the queue exceeds the set TTL value, it will become a "dead letter".
The method of setting the message TTL through the queue attribute is achieved by adding the x-message-ttl option to the args parameter of the QueueDeclare method, and the unit of this parameter is milliseconds. It can also be set via Policy or HTTP API interface.
If TTL is not set, it means that this message will not expire. If TTL is set to 0, it means that unless the message can be directly delivered to the consumer at this time, the message will be immediately discarded.
The method to set TTL for each message is to set Expiration property in the message Publishing structure of the Publish method, the unit is milliseconds. It can also be set through the HTTP API interface.
For the first method of setting the TTL attribute of the queue, once the message expires, it will be deleted from the queue. In the second method, even if the message expires, it will not be deleted immediately from the queue, because whether each message expires is Determined before delivery to consumers.
Set the TTL of the queue
x-expires option in the args parameter of the QueueDeclare method can control the maximum time that the queue is unused before it is automatically deleted. The unit is milliseconds and cannot be set to 0. Unused means that there are no consumers on the queue, the queue has not been re-declared, and the Get method has not been called within the expiration period.
Setting the TTL in the queue can be applied to a reply queue similar to RPC. In RPC, many unused queues are created.
RabbitMQ will ensure that the queue is deleted after the expiration time is reached, but it does not guarantee how timely the deletion is. After RabbitMQ restarts, the expiration time of the persistent queue will be recalculated.
Dead letter queue
DLX, the full name is Dead-Letter-Exchange, can be called a dead letter exchange. When a message becomes a dead letter in a queue, it can be re-sent to another exchange. This exchange is DLX, and the queue bound to DLX is called a dead letter queue.
Messages become dead letters generally due to the following situations:
- message was rejected (Reject/Nack), and set the requeue parameter to false.
- message has expired.
- queue reached the maximum length.
DLX is also a normal exchange, which is no different from a normal exchange. It can be specified on any queue (in fact, it is to set the attributes of a certain queue). When there is a dead letter in this queue, RabbitMQ will automatically republish the message to the set DLX, and then be routed to the dead letter queue. You can listen to the messages in this queue for corresponding processing. This feature can be used in conjunction with setting the TTL of the message to 0 to make up for the function of the immeaiate parameter.
Add DLX to this queue by setting the x-dead-letter-exchange option in the args parameter of the QueueDeclare method:
err = ch.ExchangeDeclare(
"dlx_exchange",
"direct",
true,
false,
false,
false,
nil,
)
if err != nil {
log.Fatalf("%s: %s", err, "Failed to declare an exchange")
return
}
args := make(map[string]interface{})
args["x-dead-letter-exchange"] = "dlx_exchange"
// 为队列my_queue添加DLX
q, err := ch.QueueDeclare(
"my_queue",
true,
false,
false,
false,
args,
)
if err != nil {
log.Fatalf("%s: %s", err, "Failed to declare a queue")
return
}
You can also specify a routing key for this DLX. If there is no special specification, the routing key of the original queue is used:
args["x-dead-letter-routing-key"] = "dlx_routing_key"
For RabbitMQ, DLX is a very useful feature. It can handle abnormal situations where the message cannot be correctly consumed by the consumer (the consumer calls Nack or Reject) and is placed in the dead letter queue. The subsequent analysis program can analyze the content of the dead letter queue by consuming the content of the dead letter queue. The abnormal situation encountered at that time can then improve and optimize the system.
delay queue
The object stored in the delay queue is the corresponding delayed message. The so-called "delayed message" means that when the message is sent, the consumer does not want the consumer to get the message immediately, but waits for a certain time before the consumer can get the message for consumption .
There are many usage scenarios for delay queues, such as:
- In the order system, a user usually has 30 minutes to pay after placing an order. If the payment is not successful within 30 minutes, then the order will undergo exception processing, and then the delay queue can be used to process these orders.
- The user hopes to remotely control the smart device at home through the mobile phone to work at a specified time. At this time, the user instruction can be sent to the delay queue, and the instruction will be pushed to the smart device when the set time for the instruction is up.
RabbitMQ itself does not directly support the function of the delay queue, but the function of the delay queue can be simulated through the DLX and TTL described above.
Assuming that each message needs to be set to a 10-second delay in an application, two sets of exchanges and queues can be created: regular exchange exchange.normal, regular queue queue.normal and dead letter exchange exchange.dlx, dead letter Queue queue.dlx, and then add the dead letter exchange exchange.dlx to queue.normal. The producer sends the message to exchange.normal and sets the TTL to 10 seconds, and the consumer subscribes to queue.dlx instead of queue.normal. When the message expires from queue.normal and is stored in queue.dlx, the consumer happens to consume the message with a delay of 10 seconds.
priority queue
Priority queues, as the name suggests, queues with high priority have high priority, and messages with high priority have the privilege of being consumed first.
priority queue can be implemented by setting the x-max-priority option in the args parameter of the QueueDeclare method. But this is just to configure the maximum priority of a queue. After that, you need to set the priority of the message when sending it. The setting method is the Priority property in the Publishing structure.
By default, the priority of the message is the lowest as 0, and the highest is the maximum priority set by the queue. Messages with high priority can be consumed first, but if the consumer's consumption speed is greater than the producer's speed and there is no message accumulation in the Broker, setting the priority of the sent message has no practical meaning.
RPC implementation
RPC is the abbreviation of Remote Procedure Call, that is, remote procedure call. It is a technology that requests services from a remote computer through a network without needing to understand the underlying network. The main function of RPC is to make it easier to build distributed computing, without losing the semantic simplicity of local calls while providing powerful remote call capabilities.
It is also very simple to implement RPC in RabbitMQ. client sends a request message. In order to receive the response message, we need to send a callback queue in the request message. can use the default queue:
err = ch.Publish(
"",
"rpc_queue",
false,
false,
amqp.Publishing{
ContentType: "text/plain",
CorrelationId: "",
ReplyTo: "",
Body: []byte("rpc"),
},
)
For the Publishing structure involved in the code, two of its properties are needed here:
- ReplyTo : Usually used to set up a callback queue.
- CorrelationId : used to correlate the request and the reply after calling the RPC.
It is very inefficient to create a callback queue for each RPC request as in the above code. But fortunately, there is a general solution—you can create a single callback queue for each client.
We should set a unique correlationId for each request. For the callback queue, after it receives a reply message, it can match the corresponding request according to this attribute. If the callback queue receives a reply message with an unknown correlationId, it can simply be discarded.
As shown in the figure, the processing flow of RPC is as follows:
- When the client starts, an anonymous callback queue is created.
- The client sets the replyTo and correlationId for the RPC request.
- The request is sent to the rpc_queue queue.
- The RPC server monitors the requests in the rpc_queue queue. When the request comes, the server will process and send the message with the result to the client. The received queue is the callback queue set by replyTo.
- The client listens to the callback queue. When there is a message, it checks the correlationId property, and if it matches the request, it performs other callback processing.
You can refer to the example on the RabbitMQ official website. The RPC client uses RPC to call the method of the server to get the corresponding Fibonacci value:
rpc_server.go
rpc_client.go
producer confirmation
When using RabbitMQ, after the message producer sends the message, does the message arrive at the server correctly? RabbitMQ provides two solutions to this problem:
- Transaction mechanism
- The sender confirm (publisher confirm) mechanism
Transaction mechanism
There are three methods related to the transaction mechanism in the RabbitMQ client:
func (ch *Channel) Tx() error
func (ch *Channel) TxCommit() error
func (ch *Channel) TxRollback() error
channel.Tx is used to set the current channel to transaction mode, channel.TxCommit is used to commit the transaction, and channel.TxRollback is used to roll back the transaction.
After opening the transaction through the channel.Tx method, we can publish a message to RabbitMQ. If the transaction is submitted successfully, the message must have reached RabbitMQ. If an exception is thrown due to RabbitMQ's abnormal crash or other reasons before the transaction is submitted and executed, this At that time, we can capture it, and then implement the transaction rollback by executing the channel.txRollback method.
If you want to send multiple messages, just wrap methods such as channel.Publish and channel.TxCommit into the loop.
The transaction can indeed solve the problem of message confirmation between the message sender and RabbitMQ. Only when the message is successfully received by RabbitMQ can the transaction be submitted successfully. Otherwise, the transaction can be rolled back after the exception is caught, and the message can be resent at the same time. But using the transaction mechanism will seriously affect the performance of RabbitMQ, so is there a better way? RabbitMQ provides an improved scheme, that is, the sender confirmation mechanism.
sender confirmation mechanism
producer can set the channel to confirm mode. Once the channel enters the confirm mode, all messages published on the channel will be assigned a unique ID (starting from 1). When the message is delivered to all matching queues, RabbitMQ An acknowledgment (Ack) will be sent to the producer (including the message ID), which allows the producer to know that the message has arrived at the destination correctly. If the message and queue are persistent, the confirmation message will be sent after the message is written to disk. The deliveryTag in the confirmation message sent back to the producer by RabbitMQ contains the sequence number of the confirmation message. In addition, RabbitMQ can also set the multiple parameter in the Ack method to indicate that all messages before this sequence number have been processed.
The transaction mechanism will block the sender after a message is sent to wait for RabbitMQ's response before it can continue to send the next message. In contrast, the sender confirmation mechanism is that it is asynchronous . Once a message is published, the producer application can continue to send the next message while waiting for the channel to return confirmation. When the message is finally confirmed , The producer application can process the confirmation message through the callback method. If RabbitMQ causes the message to be lost due to its own internal error, it will send a Nack command, and the producer application can also process the Nack command in the callback method.
In the confirm mode, all sent messages will be Ack or Nack once, and there will be no case that a message is both , but RabbitMQ does not make any guarantees as to how fast the message will be confirmed.
The following is a code example on the official website:
package main
import (
"log"
amqp "github.com/rabbitmq/amqp091-go"
)
func failOnError(err error, msg string) {
if err != nil {
log.Fatalf("%s: %s", msg, err)
}
}
func main() {
conn, err := amqp.Dial("amqp://guest:guest@localhost:5672/")
failOnError(err, "Failed to connect to RabbitMQ")
defer conn.Close()
ch, err := conn.Channel()
failOnError(err, "Failed to open a channel")
defer ch.Close()
// 初始化接收确认消息的通道
confirms := make(chan amqp.Confirmation)
// 注册通道使之监听发布的确认消息
ch.NotifyPublish(confirms)
go func() {
for confirm := range confirms {
// 返回确认消息
if confirm.Ack {
// code when messages is confirmed
log.Printf("Confirmed")
} else {
// code when messages is nack-ed
log.Printf("Nacked")
}
}
}()
// 将信道设置成confirm模式
err = ch.Confirm(false)
failOnError(err, "Failed to confirm")
q, err := ch.QueueDeclare(
"", // name
false, // durable
false, // delete when unused
false, // exclusive
false, // no-wait
nil, // arguments
)
failOnError(err, "Failed to declare a queue")
consume(ch, q.Name)
publish(ch, q.Name, "hello")
log.Printf(" [*] Waiting for messages. To exit press CTRL+C")
forever := make(chan bool)
<-forever
}
func consume(ch *amqp.Channel, qName string) {
msgs, err := ch.Consume(
qName, // queue
"", // consumer
true, // auto-ack
false, // exclusive
false, // no-local
false, // no-wait
nil, // args
)
failOnError(err, "Failed to register a consumer")
go func() {
for d := range msgs {
log.Printf("Received a message: %s", d.Body)
}
}()
}
func publish(ch *amqp.Channel, qName, text string) {
err := ch.Publish("", qName, false, false, amqp.Publishing{
ContentType: "text/plain",
Body: []byte(text),
})
failOnError(err, "Failed to publish a message")
}
Note: The transaction mechanism and the publisher confirm mechanism are mutually exclusive and cannot coexist.
Consumer Points
message distribution
When the RabbitMQ queue has multiple consumers, the messages received by the queue will be distributed to the consumers in a polling manner. Each message will only be sent to one consumer in the subscription list. If the load is heavier now, you only need to create more consumers to consume and process messages.
However, in many cases, the polling distribution mechanism is not so elegant. If some consumers have heavy tasks and can't consume so many messages, and some other consumers have processed the allocated messages quickly for some reasons (such as simple business logic, excellent machine performance, etc.), and the process is idle , This will cause the overall application throughput to drop.
The Qos method is used here, which can limit the maximum number of unconfirmed messages that consumers on the channel can keep:
func (ch *Channel) Qos(prefetchCount, prefetchSize int, global bool) error
- prefetchCount: The upper limit of the number of unconfirmed messages by consumers. Set to 0 means there is no upper limit.
- prefetchSize: The upper limit of the size of unconfirmed consumer messages, in B. Set to 0 means there is no upper limit.
- global: Whether to take effect globally, true means yes. global effect means that all consumers on the channel are restricted by prefetchCount and prefetchSize (otherwise only new consumers will be restricted).
Note: Qos method is invalid for pull mode.
Message sequence
RabbitMQ does not use any advanced features, there is no abnormal situation such as message loss, network failure, and there is only one consumer, and preferably only one producer, it can guarantee the order of messages. If multiple producers send messages at the same time, it is impossible to determine the order in which the messages arrive at the Broker, and the order of the messages cannot be verified.
In many cases, it will cause RabbitMQ messages to be out of order. If you want to ensure the order of messages, you need to do further processing after the business side uses RabbitMQ, such as adding a global order identifier to the message body.
**粗体** _斜体_ [链接](http://example.com) `代码` - 列表 > 引用
。你还可以使用@
来通知其他用户。