Network protocol: detailed explanation of memcached text protocol

Introduction

Anyone who has used the cache system must have heard the name of memcached. Memcached is a very good distributed memory cache system, which is widely used. Memcached is not just a web cache, it is a general data cache, basically you can store anything in memcached, its distributed design has good scalability and flexibility.

Memcached is a client-server architecture pattern. Generally speaking, after setting up the server side of Memcached on the server, you can use the client side of Memcached to exchange with the server side.

As the model of the client and the server, the communication between the two must have a specific protocol. The protocol applicable to memcached is called the memcached protocol.

There are two protocols for memcached, the text protocol and the binary protocol. This article will explain the definition of the memcached text protocol in detail.

Introduction to memcached protocol

Memcached can be regarded as a simple key-value storage system. The client requests data from the server through the key, and the server searches for the corresponding data through the hash value of the key, and then returns it to the client.

The key length in memcached generally cannot exceed 250 characters. key cannot contain control characters or whitespace characters.

In order to ensure smooth message communication between the client and the server, a special communication protocol between the client and the server is generally formulated, which is called a protocol.

What is the protocol? The protocol sounds deep and mysterious, but in fact, the protocol is the agreed message format for the interaction between the two parties.

For memcached, memcached supports both UDP and TCP protocols, and provides two protocols, namely "text protocol" and "binary protocol".

The text protocol is the protocol supported in the first version, and the binary protocol is only supported after v1.4.

Both the text protocol and the binary protocol support the same commands. The only difference between the two is that the binary protocol has lower performance latency and better scalability, while the advantage of the text protocol is that it has better debuggable performance.

The memcached text protocol contains two parts of data, lines of text and unstructured data. The former is a command from the client or a response from the server, and the latter represents the data accessed by the client. Commands end with \r\n, and data can use \r, \n or \r\n to indicate the end of the data section.

Commands supported by memcached

memcached supports three kinds of commands, which are store commands, read commands and other commands.

store command

There are a total of 6 storage commands in memcached, namely "set", "add", "replace", "append", "prepend" and "cas".

First, the client sends a command line like the following:

 command key [flags] [exptime] length [noreply]

In addition, the format of the cas command is different from the other ones:

 cas key [flags] [exptime] length [casunique] [noreply]

In the above command, command represents the name of the command, which is "set", "add", "replace", "append" and "prepend" above.

set means to set a value to the key.

Add means if the key does not exist, add it.

replace is used to replace the value of a known key.

append means appending the provided value to the value of the existing key, which is an append operation.

prepend adds the value corresponding to the current key to the provided value.

cas is an atomic operation, and only when the casunique matches, the corresponding value will be set.

flags is a very interesting parameter. This parameter is transparent to the memcached server. This parameter is only used to mark the type of the client command and will not be recognized by the server. In addition, the length of flags varies in different memcached versions. In memcached 1.2.0 or lower versions, flags is a 16-bit integer. In memcached 1.2.1 or later, flags is a 32-bit integer.

exptime is the expiration time, 0 means it will not expire.

length is the length of the value in bytes, this value does not contain the terminator "\r\n" in the value.

casunique is the unique value of a 64-bit existing entry.

noreply tells the server that this is a command that does not require reply.

After sending the command line, the client also needs to send the data block:

 <data block>\r\n

For example, we want to set the value of jack to the key of student, then the corresponding command should be as follows:

 set student 0 0 4\r\njack\r\n

The server-side response received by the corresponding client may have these values:

• "STORED\r\n", indicating successful storage.
• "NOT_STORED\r\n" means that the data was not stored successfully due to some error. This usually means that the conditions for an "add" or "replace" command are not met.
• "EXISTS\r\n" means that the value to be set has been modified since the last cas operation.
• "NOT_FOUND\r\n" means that the value to be set is used in cas.

read command

There are four read commands for memcached, namely "get", "gets", "gat" and "gats". The format of these commands is as follows:

 get <key>*\r\n
gets <key>*\r\n
gat <exptime> <key>*\r\n
gats <exptime> <key>*\r\n

Read commands in memcached do not need to be followed by additional blocks of data.

The server will query according to the received key, and each key will return a piece of data in the following format:

 VALUE <key> <flags> <bytes> [<cas unique>]\r\n
<data block>\r\n

After all data is transmitted, the server will send "END\r\n" to indicate that the transmission is complete.

The key here represents the key passed in the query.

flags is the flags passed in by the storage command.

bytes is the length of the following data block.

cas unique is the unique tag of the current item, returned in the gets or gats command.

The data block is the specific return value of the current item.

Above we mentioned 4 read commands, so what is the difference between them?

The first is the difference between get and gets. get is used to obtain the value of the key. If the key does not exist, it returns null. Multiple keys are supported. gets is used to get the value of the key with the CAS token value, if the key does not exist, it returns null. Multiple keys are supported. The difference between them is that gets will return one more cas unique value.

The difference between gat and get is that gat is a command complex of get+touch. In addition to returning the current value, it also updates the expiration time of the key.

Other commonly used commands

In addition to store and fetch, there are some other commands that are commonly used. Why are these commands called third-class commands? This is because these commands only require a single command line and do not require additional data blocks to be passed to the server.

The following is the format of the delete command:

 delete <key> [noreply]\r\n

key is the object to delete.

noreply indicates whether the return value from the server needs to be received.

There may be two corresponding server-side return values:

• "DELETED\r\n" indicates successful deletion
• "NOT_FOUND\r\n" means that the object to be deleted does not exist.

The following is the format of the Increment/Decrement command:

 incr <key> <value> [noreply]\r\n
decr <key> <value> [noreply]\r\n

key is the object to be modified.

value is the value to add or subtract, it must be a 64-bit unsigned integer.

noreply indicates whether the return value from the server needs to be received.

There are two possible returns from the server side:

• "NOT_FOUND\r\n" means that the object to be modified was not found
• "value\r\n" returns the value after successful modification

There is also a commonly used touch command to modify the key expiration time:

 touch <key> <exptime> [noreply]\r\n

key is the object to be modified.

exptime is the expiration time.

noreply indicates whether the return value from the server needs to be received.

There are two types of return values on the server side:

• "TOUCHED\r\n" means the modification is successful.
• "NOT_FOUND\r\n" means that the object to be modified does not exist.

Of course, the commands supported by memcached are far more than those mentioned above. We just picked out some of the most commonly used commands to explain.

The return value of the memcached server

The return value of the server was mentioned when explaining the specific command above. To summarize, the return value of the memcached server side is as follows:

return value	illustrate
STORED	Value stored successfully
NOT_STORED	Value store failed
EXISTS	The object to be stored in cas already exists
NOT_FOUND	The object to be modified does not exist
ERROR	An unknown command was submitted
CLIENT_ERROR errorstring	The client input is wrong, the specific error information is stored in errorstring
SERVER_ERROR errorstring	Server side exception
VALUE keys flags length	Returns the object corresponding to the key to be queried
DELETED	object has been deleted
STAT name value	Statistics
END	end of server return

Note that all return values above end with "\r\n".

Support UDP protocol

The above are the message formats of the TCP protocol. In fact memcached also supports the UDP protocol.

However, because UDP does not guarantee reliability, the use of UDP is generally used in cache query applications. Even if the query fails, it is only regarded as a cache miss, which does not affect the accuracy of the data.

In fact, the data packet format of UDP is basically the same as that of TCP, except that there is a simple frame header. And all requests must be done in a single UDP packet.

Note that only the request has this requirement here, and the server-side return does not have this restriction.

In UDP, the frame header is 8 bytes long, of which 0-1 bytes represent the request ID, which is a monotonically increasing value generated by the client. The server will use this ID to mark the response to which request. Especially if there are multiple responses on the server side.

2-3 bytes represent the serial number, and its value range is from 0 to n-1, where n is the total number of packets in the message, which is represented by 4-5 bytes.

The last 6-7 bytes are reserved for future use and are now set to 0.

Summarize

The above is an introduction to the memcached protocol. Generally speaking, we use memcached through the memcached client. If you are careful, you may find that the commands used by the client are not much different from those in the protocol. This is because The client is the encapsulation of these low-level protocols, and then exposes a simpler and easier-to-operate interface to the user.

For more information, please refer to http://www.flydean.com/23-memcached-text-protocol/

The most popular interpretation, the most profound dry goods, the most concise tutorials, and many tricks you don't know are waiting for you to discover!

Welcome to pay attention to my official account: "Program those things", understand technology, understand you better!